Incorrect wrapping of go strings

[hzarnani]

Synopsis

cgo exports Go strings as a C struct that has a pointer to the buffer and the size of that string:

typedef struct { const char *p; GoInt n; } GoString;

There are cases where swig wraps Go strings by just passing on the p field and ignoring the size field n. I reckon that's based on the (false) assumption that Go strings are null-terminated. This causes problems at least in two cases -- with string literals (constant strings that get emitted in the text section of a binary) and with calling string processors such as the strings.Split function. In both cases, the C code tries to interpret the buffer passed in much beyond the end of the actual Go string. If lucky, there's a '\0' character somewhere and the string interpretation ends without a crash. However, I have seen crashes (non-deterministically based on how Go's code generation decides to arrange string literals, or where the Go runtime environment places a Go string).

Reproducing the Bug

The following demonstrates a non-crashing example of the above-mentioned problem.

We begin by showing the version of swig/go and the platform on which it was run.

bash-3.2$ swig -version

SWIG Version 3.0.7

Compiled with clang++ [x86_64-apple-darwin15.0.0]

Configured options: +pcre

Please see http://www.swig.org for reporting bugs and further information
bash-3.2$ go version
go version go1.7.1 darwin/amd64
bash-3.2$ 

Consider a simple C++ library exported in header bam.hpp that defines a function bam which takes as input a C string and prints it to standard output. We define a swig interface main_simple.i that includes the entire header and generates wrappers. (We also define main_alloc_str.i which is used later to show that it solves the problem.) Finally, a Go program is defined that tokenizes the first argument to the program (with comma as delimiter) and passes the first token to bam to print it. (If no argument is specified, a default string "foo,bar,bam" is used.) Below is the source code. Note that the only difference between the two *.i files is that the longer one extends the simpler one by overriding how char * arguments are dealt with.

bash-3.2$ ls
bam.hpp			main_alloc_str.i
call_bam.go		main_simple.i
bash-3.2$ 

bam.hpp contains:

#pragma once

#include <iostream>

static inline void bam(char const * const str) {
  std::cout << str << std::endl;
}

main_simple.i contains:

%module main
%{
#include "bam.hpp"
%}
%include "bam.hpp"

call_bam.go contains:

package main

import (
	"os"
	"strings"
)

func main() {
	csv := "foo,bar,bam"
	if len(os.Args) > 1 {
		csv = os.Args[1]
	}
	Bam(strings.Split(csv, ",")[0])
}

The following is expected to print only 1st on the screen but prints the whole argument.

bash-3.2$ EXE=/tmp/go-str-demo
bash-3.2$ ln -fs main_simple.i main.swigcxx
bash-3.2$ eval $(go build -work -o $EXE 2>&1) && $EXE 1st,2nd,3rd
1st,2nd,3rd
bash-3.2$ 

That's because in the Go program, csv's p points to a buffer "1st,2nd,3rd" with an n value of 11, and the output of strings.Split points to the same buffer (has the same p as that of csv) and only differs in its n -- 3.

The problem is much worse if a string literal in the program is passed to bam.

bash-3.2$ eval $(go build -work -o $EXE 2>&1) && $EXE
foo,bar,bamgccheckmarkgcscanvalidlfstackpushlost mcachemach error scheddetailshort writetracealloc(unreachablework.nproc=work.nwait= [recovered] allocCount  m->gsignal= pages/byte
 preemptoff= s.elemsize= span.limit= span.state= sysmonwait=) p->status=, consumed: , gp.stack=[, released: -byte limit
Bidi_ControlI/O possibleJoin_ControlMeetei_MayekPahawh_HmongSora_SompengSyloti_Nagriabi mismatchbad GC phasebad g statusbad pagesizebad recoverybut progSizecan't happencas64 failedchan receivechild exiteddevice errorend tracegc
entersyscallgcpacertracehost is downillegal seeknot a STREAMout of rangereleasep: m=runtime: gp=runtime: pc=short buffer MB released
 bytesMarked= called from  idlethreads= is nil, not  s.sizeclass= span.base()= syscalltick= with GC prog, gp->status=, not pointer-byte block (/usr/local/goGC sweep waitMende_KikakuiOld_HungarianSIGKILL: killSIGQUIT: quitSIGSTOP: stop	morebuf={pc:bad hmap sizebad map statefatal error: kevent failedload64 failednil stackbaseout of memoryresource busys.allocCount=still in listvalue method xadd64 failedxchg64 failed nmidlelocked= to finalizer  untyped args -thread limit
GC assist waitGC worker initMB; allocated Other_ID_StartPattern_SyntaxQuotation_MarkSIGABRT: abortallocfreetracebad allocCountbad span statebad stack sizedirect convT2Edirect convT2Ifile too largefinalizer waitgcstoptheworldis a directorykern.osreleasekey size wrongmemprofileratenil elem type!no module datanot in ranges:protocol errorsemaphore_waitstack overflowstopm spinningstore64 failedtext file busytoo many linkstoo many userstrace/BPT trapunexpected EOFwork.full != 0, t.hmap.size =ASCII_Hex_DigitOther_LowercaseOther_UppercasePsalter_Pahlaviacquirep: p->m=bad system callforce gc (idle)malloc deadlockmissing mcache?ms: gomaxprocs=network is downno such processnon-Go functionnot a directorynot implementedrecovery failedruntime error: runtime.gopanicruntime: frame scan missed a gstartm: m has pstopm holding p NumSizeClasses= already; errno= but memory size mheap.sweepgen= untyped locals , not a function0123456789abcdefGC worker (idle)Imperial_AramaicMSpanList_InsertMSpanList_RemoveMeroitic_CursiveOther_AlphabeticSIGNONE: no trap
runtime stack:
bad frame layoutbad special kindbad symbol tablecastogscanstatusgc: unswept spangcshrinkstackoffinteger overflowinvalid argumentinvalid spdelta message too longmissing stackmapno route to hostpacer: H_m_prev=policy not foundreflect mismatchresult too largeruntime:  g:  g=runtime: gp: gp=runtime: nameOffruntime: nelems=runtime: textOffruntime: typeOffsemaphore_createsemaphore_signaltime: bad [0-9]*to non-Go memoryvalue size wrongworkbuf is empty initialHeapLive= spinningthreads=: missing method Old_North_ArabianOld_South_ArabianOther_ID_ContinueRPC struct is badRPC version wrongSIGBUS: bus errorSIGINT: interruptSentence_TerminalUnified_Ideograph[string too long]bad TinySizeClassbad dequeueRescanbad stackfreelistcross-device linkentersyscallblockexec format errorgcstackbarrierallgcstackbarrieroffkey align too bigmark - bad statusmarkBits overflownext_gc underflownotetsleepg on g0permission deniedruntime.newosprocruntime: pointer scanobject n == 0thread exhaustionunknown caller pc because dotdotdot, locked to threadCaucasian_AlbanianENOLINK (Reserved)Variation_Selectorbad NumSizeClassesconnection refusedfile name too longforEachP: not donegarbage collectiongcBlackenPromptly=identifier removedindex out of rangeinput/output errorneed authenticatorno child processesno locks availableoperation canceledsegmentation faultstack barrier lostsuspended (signal)too many processes called using nil *,  g->atomicstatus=, gp->atomicstatus=/usr/lib/locale/TZ/Canadian_AboriginalIDS_Binary_OperatorMB of spans; swept MSpan_Sweep: state=Pattern_White_SpaceRPC prog. not availSIGTRAP: trace trap_cgo_setenv missingattribute not foundbad file descriptorbad kind in runfinqbad notifyList sizebad runtime·mstartbad value for fieldcgocall unavailabledevice power is offdirectory not emptydisc quota exceededfile already existsfile does not existillegal instructionincorrect alignmentinformation requestmarkroot: bad indexno STREAM resourcesnotesleep not on g0nwait > work.nprocsoperation timed outoverflow is not nilpanic during mallocpanic during panic
panic holding locksprevious owner diedruntime: pcdata is runtime: preempt g0runtime: sizeclass=stopm holding locksstopped (tty input)too many open filesvalue align too bigwindow size changeswrite of Go pointer markroot jobs done
 to unallocated span/usr/share/zoneinfo/: leftover defer sp=EMULTIHOP (Reserved)Egyptian_HieroglyphsIDS_Trinary_OperatorMSpanList_InsertBackMeroitic_HieroglyphsSIGALRM: alarm clockSIGTERM: terminationSTREAM ioctl timeoutSeek: invalid offsetSeek: invalid whenceTerminal_Punctuationauthentication errorbad defer size classbad use of bucket.bpbad use of bucket.mpchan send (nil chan)close of nil channelfloating point errorforcegc: phase errorgo of nil func valuegopark: bad g statusinvalid itab lockinginvalid m->locked = malloc during signalmissed stack barriernotetsleep not on g0pacer: assist ratio=preempt off reason: reflect.makeFuncStubrescan list overflowruntime: casgstatus runtime: unknown pc runtime: work.nwait=stopped (tty output)time: invalid numbertrace: out of memoryurgent I/O conditionworkbuf is not empty gp.gcscanvalid=true
 previous allocCount=Anatolian_HieroglyphsDYLD_INSERT_LIBRARIESInscriptional_PahlaviOther_Grapheme_Extend_cgo_unsetenv missingblock device requiredcheckdead: runnable gconcurrent map writesdefer on system stackdevice not configuredfindrunnable: wrong pgcprocs inconsistencyillegal byte sequenceinvalid stack pointermalformed Mach-o filepanic on system stackprogram version wrongread-only file systemreleasep: invalid argruntime: confused by runtime: kevent on fdruntime: newstack sp=runtime: work.nwait =stale NFS file handlestartlockedm: m has pstartm: m is spinningstate not recoverablestopg: invalid statususer defined signal 1user defined signal 2virtual timer expired/lib/time/zoneinfo.zipInscriptional_ParthianSIGTSTP: keyboard stopaddress already in useargument list too longassembly checks failedbad g->status in readycannot allocate memorycputime limit exceededfreeIndex is not validgetenv before env initinteger divide by zerointerface conversion: minpc or maxpc invalidnetwork is unreachablenon-Go function at pc=panic during softfloatprotocol not availableprotocol not supportedruntime.main not on m0runtime: invalid type runtime: out of memoryruntime: s.allocCount=runtime:scanstack: gp=s.freeindex > s.nelemsscanstack - bad statussend on closed channelspan has no free spacestack not a power of 2timer goroutine (idle)trace: alloc too large is not in the Go heap
) not in usable range [Logical_Order_ExceptionNoncharacter_Code_PointSIGIO: i/o now possibleSIGSYS: bad system callacquirep: already in gochan receive (nil chan)close of closed channelfilesize limit exceededgarbage collection scangcDrain phase incorrectinterrupted system callleft over markroot jobsmakechan: bad alignmentmissing type in runfinqno space left on deviceoperation not permittedoperation not supportedpanic during preemptoffprocresize: invalid argprofiling timer expiredreflect.methodValueCallruntime: internal errorruntime: sizeof(hmap) =s.allocCount > s.nelemsschedule: holding lockssocket is not connectedspan has no free stacksstack growth after forkunexpected signal valuework.nwait > work.nproc/usr/share/lib/zoneinfo/bad defer entry in panicbad defer size class: i=can't scan our own stackconnection reset by peerfloating point exceptionfunction not implementedg already on rescan listgcDrainN phase incorrecthash of unhashable type runqsteal: runq overflowruntime: found obj at *(runtime: misaligned funcscanstack in wrong phasespan has no free objectsspanBytesAlloc underflowstack trace unavailable
 to unused region of span_cgo_thread_start missingacquirep: invalid p stateallgadd: bad status Gidlebad procedure for programbad status in shrinkstackcan't scan gchelper stackchansend: spurious wakeupcheckdead: no m for timercheckdead: no p for timermach_semcreate desc countmissing stack in newstackno buffer space availableno such file or directoryoperation now in progressreleasep: invalid p stateresource deadlock avoidedruntime: program exceeds runtime·lock: lock countslice bounds out of rangesocket type not supportedstartm: p has runnable gsstopTheWorld: not stoppedstoplockedm: not runnableunexpected fault address : leftover stack barriers bad CPU type in executableentersyscall inconsistent forEachP: P did not run fnfreeSpan given cached spanfreedefer with d.fn != nilinitSpan: unaligned lengthmakemap: size out of rangeno message of desired typenotewakeup - double wakeupout of memory (stackalloc)persistentalloc: size == 0runtime: writebarrierptr *startlockedm: locked to me expected stack barrier PC G waiting list is corruptedMSpan_Sweep: bad span stateSIGILL: illegal instructionSIGXCPU: cpu limit exceededcgoUse should not be calledevacuation not done in timefailed MSpanList_InsertBackgcstopm: not waiting for gcgrowslice: cap out of rangeinternal lockOSThread errorinvalid profile bucket typemakechan: size out of rangemakeslice: cap out of rangemakeslice: len out of rangeprogToPointerMask: overflowrestartg: unexpected statusruntime: checkdead: find g runtime: checkdead: nmidle=runtime·unlock: lock countscanframe: bad symbol tablesigsend: inconsistent statesocket is already connectedstack size not a power of 2startm: negative nmspinningstopTheWorld: holding lockswork.nwait was > work.nproc args stack map entries for FixedStack is not power-of-2MHeap_FreeSpanLocked - span MSpan_Sweep: m is not lockedPrepended_Concatenation_MarkSIGCONT: continue after stopSIGHUP: terminal line hangupSIGWINCH: window size change_mheap_alloc not on g0 stackbad pointer in write barriercan't preserve unlinked spancomparing uncomparable type destination address requiredfindrunnable: netpoll with pfound next stack barrier at gchelperstart: bad m->helpgcgcstopm: negative nmspinninginvalid runtime symbol tablemissing stack in shrinkstackneed padding in bucket (key)newproc1: new g is not Gdeadnewproc1: newg missing stackos: process already finishedruntime: casgstatus: oldval=runtime: no module data for runtime: p.gcMarkWorkerMode=runtime: stat overflow: val size is 0 but class is not 0SIGPIPE: write to broken pipeabi mismatch detected betweenaddspecial on invalid pointergc done but gcphase != _GCoffgfput: bad status (not Gdead)invalid length of trace eventio: read/write on closed pipenotesleep - waitm out of syncoperation already in progressprotocol family not supportedruntime: bad g in cgocallbackruntime: impossible type kindruntime: split stack overflowruntime: stat underflow: val runtime: sudog with non-nil cruntime: unknown pc in defer semacquire not on the G stackstring concatenation too longtoo many open files in systemtraceback has leftover defers locals stack map entries for MHeap_AllocLocked - bad npagesSIGPROF: profiling alarm clockSIGUSR1: user-defined signal 1SIGUSR2: user-defined signal 2SIGVTALRM: virtual alarm clockassignment to entry in nil mapcan't assign requested addresscheckdead: inconsistent countsfreedefer with d._panic != nilinappropriate ioctl for deviceinvalid function symbol table
need padding in bucket (value)no message available on STREAMnotetsleep - waitm out of syncprotocol wrong type for socketrunqputslow: queue is not fullruntime: bad pointer in frame runtime: found in object at *(socket operation on non-socketspan contains too many objectssync: inconsistent mutex statesync: unlock of unlocked mutex...additional frames elided...
SIGSEGV: segmentation violationcasgstatus: bad incoming valuescheckmark found unmarked objectentersyscallblock inconsistent findfunc: bad findfunctab entryinternal error - misuse of itabmalformed time zone informationmheap_freestack not on g0 stackpacer: sweep done at heap size resetspinning: not a spinning mruntime: cannot allocate memoryruntime: split stack overflow: shared library version mismatchSIGFPE: floating-point exceptionSIGTTOU: background write to ttycan't send after socket shutdowngchelper not running on g0 stackmheap_allocstack not on g0 stacknon-Go code disabled sigaltstacknumerical argument out of domainremovespecial on invalid pointerresource temporarily unavailableruntime: failed MSpanList_Insertruntime: failed MSpanList_Removeruntime: gcBitsArenas.next.free=runtime: mcall function returnedruntime: newstack called from g=runtime: stack split at bad timeruntime: sudog with non-nil elemruntime: sudog with non-nil nextruntime: sudog with non-nil prevscanstack: goroutine not stoppedsoftware caused connection abortstackmapdata: index out of rangesweep increased allocation countSIGCHLD: child status has changedSIGTTIN: background read from ttySIGXFSZ: file size limit exceededconcurrent map read and map writefindrunnable: negative nmspinningfreeing stack not in a stack spanheapBitsSetType: unexpected shiftinappropriate file type or formatoperation not supported by deviceoperation not supported on socketruntime: bsdthread_register errorruntime: castogscanstatus oldval=runtime: goroutine stack exceeds runtime: memory allocated by OS (runtime: name offset out of rangeruntime: text offset out of rangeruntime: type offset out of rangestackalloc not on scheduler stackstoplockedm: inconsistent lockingsync: RUnlock of unlocked RWMutextoo many levels of remote in pathtoo many levels of symbolic linkstoo many references: can't spliceMHeap_AllocLocked - MSpan not freeMSpan_EnsureSwept: m is not lockedOther_Default_Ignorable_Code_PointSIGURG: urgent condition on socketbad executable (or shared library)forEachP: sched.safePointWait != 0heapBitsForSpan: base out of rangeinconsistent state in stackBarrierruntime.SetFinalizer: cannot pass runtime: g is running but p is notruntime: misuse of rewindmorestackruntime: unexpected return pc for schedule: spinning with local workMHeap_FreeSpanLocked - invalid freefindrunnable: netpoll with spinninggreyobject: obj not pointer-alignedmisrounded allocation in mallocinitnetwork dropped connection on resetpersistentalloc: align is too largepidleput: P has non-empty run queuetraceback did not unwind completelyGo pointer stored into non-Go memoryMStats vs MemStatsType size mismatchSIGEMT: emulate instruction executedkey size not a multiple of key alignmarkrootSpans during second markrootruntime: bad notifyList size - sync=runtime: invalid pc-encoded table f=runtime: invalid typeBitsBulkBarrierruntime: mcall called on m->g0 stackruntime: sudog with non-nil waitlinkruntime: wrong goroutine in newstacksignal arrived during cgo execution
uncaching span but s.allocCount == 0SIGINFO: status request from keyboard_cgo_notify_runtime_init_done missingall goroutines are asleep - deadlock!attempt to execute C code on Go stackruntime: allocation size out of rangesetprofilebucket: profile already setstartTheWorld: inconsistent mp->nextptraceback has leftover stack barriersarg size to reflect.call more than 1GBgcBgMarkWorker: blackening not enabledmakechan: invalid channel element typeruntime: function symbol table header:runtime: sudog with non-nil selectdoneMSpan_Sweep: bad span state after sweepheapBitsSetTypeGCProg: small allocationmisrounded allocation in MHeap_SysAllocreceived on thread with no signal stackruntime: casfrom_Gscanstatus failed gp=runtime: typeBitsBulkBarrier with type stack growth not allowed in system calldeferproc: d.panic != nil after newdeferheapBitsBulkBarrier: unaligned argumentsreceived but handler not on signal stackrefill of span with free space remainingruntime.SetFinalizer: first argument is runtime: out of memory: cannot allocate value size not a multiple of value alignMHeap_FreeSpanLocked - invalid span stateMHeap_FreeSpanLocked - invalid stack freegcSweep being done but phase is not GCoffruntime.SetFinalizer: second argument is runtime.makemap: unsupported map key typeruntime: typeBitsBulkBarrier without typesetCheckmarked and isCheckmarked disagreevalue too large to be stored in data typeacquireSudog: found s.elem != nil in cachepersistentalloc: align is not a power of 2runtime: heapBitsSetTypeGCProg: total bitsunexpected signal during runtime executiongcBgMarkWorker: unexpected gcMarkWorkerModegentraceback before goexitPC initializationheapBitsSetTypeGCProg: unexpected bit countmultiple Read calls return no data or errorruntime.SetFinalizer: finalizer already setruntime.SetFinalizer: first argument is nilruntime: casfrom_Gscanstatus bad oldval gp=runtime: lfstackpush invalid packing: node=runtime: releaseSudog with non-nil gp.paramunknown runnable goroutine during bootstrapgcmarknewobject called while doing checkmarksystemstack called from unexpected goroutineexitsyscall: syscall frame is no longer validheapBitsSetType: called with non-pointer typeruntime: failed to create new OS thread (have runtime: name offset base pointer out of rangeruntime: panic before malloc heap initialized
runtime: text offset base pointer out of rangeruntime: type offset base pointer out of rangeP has cached GC work at end of mark terminationaddress family not supported by protocol familyP has unflushed stats at end of mark terminationruntime: cannot map pages in arena address spacewritebarrierptr_nostore1 called with mp.p == nilcasgstatus: waiting for Gwaiting but is Grunnableinvalid memory address or nil pointer dereferenceruntime: unexpected waitm - semaphore out of syncs.allocCount != s.nelems && freeIndex == s.nelemsmallocgc called with gcphase == _GCmarkterminationrecursive call during initialization - linker skewruntime: unable to acquire - semaphore out of syncruntime: cannot reserve arena virtual address spacecasfrom_Gscanstatus: gp->status is not in scan statefunction symbol table not sorted by program counter:runtime.SetFinalizer: pointer not in allocated blockruntime: use of FixAlloc_Alloc before FixAlloc_Init
runtime: failed to unwind through stackBarrier at SP 	goroutine running on other thread; stack unavailable
gcControllerState.findRunnable: blackening not enabledno goroutines (main called runtime.Goexit) - deadlock!casfrom_Gscanstatus:top gp->status is not in scan statenewproc: function arguments too large for new goroutinein gcMark expecting to see gcphase as _GCmarkterminationgentraceback cannot trace user goroutine on its own stacknon-Go code set up signal handler without SA_ONSTACK flagruntime:stoplockedm: g is not Grunnable or Gscanrunnable
found bad pointer in Go heap (incorrect use of unsafe or cgo?)runtime: internal error: misuse of lockOSThread/unlockOSThreadruntime: bsdthread_register error (unset DYLD_INSERT_LIBRARIES)runtime.SetFinalizer: pointer not at beginning of allocated blockruntime:greyobject: checkmarks finds unexpected unmarked object obj=runtime: found space for saved base pointer, but no framepointer experiment

bash-3.2$ 

As can be seen, there was a very large, multi-line string constant in which "foo,bar,bam" was contained, and there's a lot of stuff that follows it.

bash-3.2$ strings $EXE | grep -A24 'foo,bar,bam'
sched={pc:alarm clockbad addressbad messagebad timedivbroken pipecgocall nilcreated by errorStringfailed withfile existsfloat32nan2float64nan1float64nan2float64nan3foo,bar,bamgccheckmarkgcscanvalidlfstackpushlost mcachemach error scheddetailshort writetracealloc(unreachablework.nproc=work.nwait= [recovered] allocCount  m->gsignal= pages/byte
 preemptoff= s.elemsize= span.limit= span.state= sysmonwait=) p->status=, consumed: , gp.stack=[, released: -byte limit
Bidi_ControlI/O possibleJoin_ControlMeetei_MayekPahawh_HmongSora_SompengSyloti_Nagriabi mismatchbad GC phasebad g statusbad pagesizebad recoverybut progSizecan't happencas64 failedchan receivechild exiteddevice errorend tracegc
entersyscallgcpacertracehost is downillegal seeknot a STREAMout of rangereleasep: m=runtime: gp=runtime: pc=short buffer MB released
 bytesMarked= called from  idlethreads= is nil, not  s.sizeclass= span.base()= syscalltick= with GC prog, gp->status=, not pointer-byte block (/usr/local/goGC sweep waitMende_KikakuiOld_HungarianSIGKILL: killSIGQUIT: quitSIGSTOP: stop
morebuf={pc:bad hmap sizebad map statefatal error: kevent failedload64 failednil stackbaseout of memoryresource busys.allocCount=still in listvalue method xadd64 failedxchg64 failed nmidlelocked= to finalizer  untyped args -thread limit
GC assist waitGC worker initMB; allocated Other_ID_StartPattern_SyntaxQuotation_MarkSIGABRT: abortallocfreetracebad allocCountbad span statebad stack sizedirect convT2Edirect convT2Ifile too largefinalizer waitgcstoptheworldis a directorykern.osreleasekey size wrongmemprofileratenil elem type!no module datanot in ranges:protocol errorsemaphore_waitstack overflowstopm spinningstore64 failedtext file busytoo many linkstoo many userstrace/BPT trapunexpected EOFwork.full != 0, t.hmap.size =ASCII_Hex_DigitOther_LowercaseOther_UppercasePsalter_Pahlaviacquirep: p->m=bad system callforce gc (idle)malloc deadlockmissing mcache?ms: gomaxprocs=network is downno such processnon-Go functionnot a directorynot implementedrecovery failedruntime error: runtime.gopanicruntime: frame scan missed a gstartm: m has pstopm holding p NumSizeClasses= already; errno= but memory size mheap.sweepgen= untyped locals , not a function0123456789abcdefGC worker (idle)Imperial_AramaicMSpanList_InsertMSpanList_RemoveMeroitic_CursiveOther_AlphabeticSIGNONE: no trap
runtime stack:
bad frame layoutbad special kindbad symbol tablecastogscanstatusgc: unswept spangcshrinkstackoffinteger overflowinvalid argumentinvalid spdelta message too longmissing stackmapno route to hostpacer: H_m_prev=policy not foundreflect mismatchresult too largeruntime:  g:  g=runtime: gp: gp=runtime: nameOffruntime: nelems=runtime: textOffruntime: typeOffsemaphore_createsemaphore_signaltime: bad [0-9]*to non-Go memoryvalue size wrongworkbuf is empty initialHeapLive= spinningthreads=: missing method Old_North_ArabianOld_South_ArabianOther_ID_ContinueRPC struct is badRPC version wrongSIGBUS: bus errorSIGINT: interruptSentence_TerminalUnified_Ideograph[string too long]bad TinySizeClassbad dequeueRescanbad stackfreelistcross-device linkentersyscallblockexec format errorgcstackbarrierallgcstackbarrieroffkey align too bigmark - bad statusmarkBits overflownext_gc underflownotetsleepg on g0permission deniedruntime.newosprocruntime: pointer scanobject n == 0thread exhaustionunknown caller pc because dotdotdot, locked to threadCaucasian_AlbanianENOLINK (Reserved)Variation_Selectorbad NumSizeClassesconnection refusedfile name too longforEachP: not donegarbage collectiongcBlackenPromptly=identifier removedindex out of rangeinput/output errorneed authenticatorno child processesno locks availableoperation canceledsegmentation faultstack barrier lostsuspended (signal)too many processes called using nil *,  g->atomicstatus=, gp->atomicstatus=/usr/lib/locale/TZ/Canadian_AboriginalIDS_Binary_OperatorMB of spans; swept MSpan_Sweep: state=Pattern_White_SpaceRPC prog. not availSIGTRAP: trace trap_cgo_setenv missingattribute not foundbad file descriptorbad kind in runfinqbad notifyList sizebad runtime
mstartbad value for fieldcgocall unavailabledevice power is offdirectory not emptydisc quota exceededfile already existsfile does not existillegal instructionincorrect alignmentinformation requestmarkroot: bad indexno STREAM resourcesnotesleep not on g0nwait > work.nprocsoperation timed outoverflow is not nilpanic during mallocpanic during panic
panic holding locksprevious owner diedruntime: pcdata is runtime: preempt g0runtime: sizeclass=stopm holding locksstopped (tty input)too many open filesvalue align too bigwindow size changeswrite of Go pointer markroot jobs done
 to unallocated span/usr/share/zoneinfo/: leftover defer sp=EMULTIHOP (Reserved)Egyptian_HieroglyphsIDS_Trinary_OperatorMSpanList_InsertBackMeroitic_HieroglyphsSIGALRM: alarm clockSIGTERM: terminationSTREAM ioctl timeoutSeek: invalid offsetSeek: invalid whenceTerminal_Punctuationauthentication errorbad defer size classbad use of bucket.bpbad use of bucket.mpchan send (nil chan)close of nil channelfloating point errorforcegc: phase errorgo of nil func valuegopark: bad g statusinvalid itab lockinginvalid m->locked = malloc during signalmissed stack barriernotetsleep not on g0pacer: assist ratio=preempt off reason: reflect.makeFuncStubrescan list overflowruntime: casgstatus runtime: unknown pc runtime: work.nwait=stopped (tty output)time: invalid numbertrace: out of memoryurgent I/O conditionworkbuf is not empty gp.gcscanvalid=true
 previous allocCount=Anatolian_HieroglyphsDYLD_INSERT_LIBRARIESInscriptional_PahlaviOther_Grapheme_Extend_cgo_unsetenv missingblock device requiredcheckdead: runnable gconcurrent map writesdefer on system stackdevice not configuredfindrunnable: wrong pgcprocs inconsistencyillegal byte sequenceinvalid stack pointermalformed Mach-o filepanic on system stackprogram version wrongread-only file systemreleasep: invalid argruntime: confused by runtime: kevent on fdruntime: newstack sp=runtime: work.nwait =stale NFS file handlestartlockedm: m has pstartm: m is spinningstate not recoverablestopg: invalid statususer defined signal 1user defined signal 2virtual timer expired/lib/time/zoneinfo.zipInscriptional_ParthianSIGTSTP: keyboard stopaddress already in useargument list too longassembly checks failedbad g->status in readycannot allocate memorycputime limit exceededfreeIndex is not validgetenv before env initinteger divide by zerointerface conversion: minpc or maxpc invalidnetwork is unreachablenon-Go function at pc=panic during softfloatprotocol not availableprotocol not supportedruntime.main not on m0runtime: invalid type runtime: out of memoryruntime: s.allocCount=runtime:scanstack: gp=s.freeindex > s.nelemsscanstack - bad statussend on closed channelspan has no free spacestack not a power of 2timer goroutine (idle)trace: alloc too large is not in the Go heap
) not in usable range [Logical_Order_ExceptionNoncharacter_Code_PointSIGIO: i/o now possibleSIGSYS: bad system callacquirep: already in gochan receive (nil chan)close of closed channelfilesize limit exceededgarbage collection scangcDrain phase incorrectinterrupted system callleft over markroot jobsmakechan: bad alignmentmissing type in runfinqno space left on deviceoperation not permittedoperation not supportedpanic during preemptoffprocresize: invalid argprofiling timer expiredreflect.methodValueCallruntime: internal errorruntime: sizeof(hmap) =s.allocCount > s.nelemsschedule: holding lockssocket is not connectedspan has no free stacksstack growth after forkunexpected signal valuework.nwait > work.nproc/usr/share/lib/zoneinfo/bad defer entry in panicbad defer size class: i=can't scan our own stackconnection reset by peerfloating point exceptionfunction not implementedg already on rescan listgcDrainN phase incorrecthash of unhashable type runqsteal: runq overflowruntime: found obj at *(runtime: misaligned funcscanstack in wrong phasespan has no free objectsspanBytesAlloc underflowstack trace unavailable
 to unused region of span_cgo_thread_start missingacquirep: invalid p stateallgadd: bad status Gidlebad procedure for programbad status in shrinkstackcan't scan gchelper stackchansend: spurious wakeupcheckdead: no m for timercheckdead: no p for timermach_semcreate desc countmissing stack in newstackno buffer space availableno such file or directoryoperation now in progressreleasep: invalid p stateresource deadlock avoidedruntime: program exceeds runtime
lock: lock countslice bounds out of rangesocket type not supportedstartm: p has runnable gsstopTheWorld: not stoppedstoplockedm: not runnableunexpected fault address : leftover stack barriers bad CPU type in executableentersyscall inconsistent forEachP: P did not run fnfreeSpan given cached spanfreedefer with d.fn != nilinitSpan: unaligned lengthmakemap: size out of rangeno message of desired typenotewakeup - double wakeupout of memory (stackalloc)persistentalloc: size == 0runtime: writebarrierptr *startlockedm: locked to me expected stack barrier PC G waiting list is corruptedMSpan_Sweep: bad span stateSIGILL: illegal instructionSIGXCPU: cpu limit exceededcgoUse should not be calledevacuation not done in timefailed MSpanList_InsertBackgcstopm: not waiting for gcgrowslice: cap out of rangeinternal lockOSThread errorinvalid profile bucket typemakechan: size out of rangemakeslice: cap out of rangemakeslice: len out of rangeprogToPointerMask: overflowrestartg: unexpected statusruntime: checkdead: find g runtime: checkdead: nmidle=runtime
unlock: lock countscanframe: bad symbol tablesigsend: inconsistent statesocket is already connectedstack size not a power of 2startm: negative nmspinningstopTheWorld: holding lockswork.nwait was > work.nproc args stack map entries for FixedStack is not power-of-2MHeap_FreeSpanLocked - span MSpan_Sweep: m is not lockedPrepended_Concatenation_MarkSIGCONT: continue after stopSIGHUP: terminal line hangupSIGWINCH: window size change_mheap_alloc not on g0 stackbad pointer in write barriercan't preserve unlinked spancomparing uncomparable type destination address requiredfindrunnable: netpoll with pfound next stack barrier at gchelperstart: bad m->helpgcgcstopm: negative nmspinninginvalid runtime symbol tablemissing stack in shrinkstackneed padding in bucket (key)newproc1: new g is not Gdeadnewproc1: newg missing stackos: process already finishedruntime: casgstatus: oldval=runtime: no module data for runtime: p.gcMarkWorkerMode=runtime: stat overflow: val size is 0 but class is not 0SIGPIPE: write to broken pipeabi mismatch detected betweenaddspecial on invalid pointergc done but gcphase != _GCoffgfput: bad status (not Gdead)invalid length of trace eventio: read/write on closed pipenotesleep - waitm out of syncoperation already in progressprotocol family not supportedruntime: bad g in cgocallbackruntime: impossible type kindruntime: split stack overflowruntime: stat underflow: val runtime: sudog with non-nil cruntime: unknown pc in defer semacquire not on the G stackstring concatenation too longtoo many open files in systemtraceback has leftover defers locals stack map entries for MHeap_AllocLocked - bad npagesSIGPROF: profiling alarm clockSIGUSR1: user-defined signal 1SIGUSR2: user-defined signal 2SIGVTALRM: virtual alarm clockassignment to entry in nil mapcan't assign requested addresscheckdead: inconsistent countsfreedefer with d._panic != nilinappropriate ioctl for deviceinvalid function symbol table
need padding in bucket (value)no message available on STREAMnotetsleep - waitm out of syncprotocol wrong type for socketrunqputslow: queue is not fullruntime: bad pointer in frame runtime: found in object at *(socket operation on non-socketspan contains too many objectssync: inconsistent mutex statesync: unlock of unlocked mutex...additional frames elided...
SIGSEGV: segmentation violationcasgstatus: bad incoming valuescheckmark found unmarked objectentersyscallblock inconsistent findfunc: bad findfunctab entryinternal error - misuse of itabmalformed time zone informationmheap_freestack not on g0 stackpacer: sweep done at heap size resetspinning: not a spinning mruntime: cannot allocate memoryruntime: split stack overflow: shared library version mismatchSIGFPE: floating-point exceptionSIGTTOU: background write to ttycan't send after socket shutdowngchelper not running on g0 stackmheap_allocstack not on g0 stacknon-Go code disabled sigaltstacknumerical argument out of domainremovespecial on invalid pointerresource temporarily unavailableruntime: failed MSpanList_Insertruntime: failed MSpanList_Removeruntime: gcBitsArenas.next.free=runtime: mcall function returnedruntime: newstack called from g=runtime: stack split at bad timeruntime: sudog with non-nil elemruntime: sudog with non-nil nextruntime: sudog with non-nil prevscanstack: goroutine not stoppedsoftware caused connection abortstackmapdata: index out of rangesweep increased allocation countSIGCHLD: child status has changedSIGTTIN: background read from ttySIGXFSZ: file size limit exceededconcurrent map read and map writefindrunnable: negative nmspinningfreeing stack not in a stack spanheapBitsSetType: unexpected shiftinappropriate file type or formatoperation not supported by deviceoperation not supported on socketruntime: bsdthread_register errorruntime: castogscanstatus oldval=runtime: goroutine stack exceeds runtime: memory allocated by OS (runtime: name offset out of rangeruntime: text offset out of rangeruntime: type offset out of rangestackalloc not on scheduler stackstoplockedm: inconsistent lockingsync: RUnlock of unlocked RWMutextoo many levels of remote in pathtoo many levels of symbolic linkstoo many references: can't spliceMHeap_AllocLocked - MSpan not freeMSpan_EnsureSwept: m is not lockedOther_Default_Ignorable_Code_PointSIGURG: urgent condition on socketbad executable (or shared library)forEachP: sched.safePointWait != 0heapBitsForSpan: base out of rangeinconsistent state in stackBarrierruntime.SetFinalizer: cannot pass runtime: g is running but p is notruntime: misuse of rewindmorestackruntime: unexpected return pc for schedule: spinning with local workMHeap_FreeSpanLocked - invalid freefindrunnable: netpoll with spinninggreyobject: obj not pointer-alignedmisrounded allocation in mallocinitnetwork dropped connection on resetpersistentalloc: align is too largepidleput: P has non-empty run queuetraceback did not unwind completelyGo pointer stored into non-Go memoryMStats vs MemStatsType size mismatchSIGEMT: emulate instruction executedkey size not a multiple of key alignmarkrootSpans during second markrootruntime: bad notifyList size - sync=runtime: invalid pc-encoded table f=runtime: invalid typeBitsBulkBarrierruntime: mcall called on m->g0 stackruntime: sudog with non-nil waitlinkruntime: wrong goroutine in newstacksignal arrived during cgo execution
uncaching span but s.allocCount == 0SIGINFO: status request from keyboard_cgo_notify_runtime_init_done missingall goroutines are asleep - deadlock!attempt to execute C code on Go stackruntime: allocation size out of rangesetprofilebucket: profile already setstartTheWorld: inconsistent mp->nextptraceback has leftover stack barriersarg size to reflect.call more than 1GBgcBgMarkWorker: blackening not enabledmakechan: invalid channel element typeruntime: function symbol table header:runtime: sudog with non-nil selectdoneMSpan_Sweep: bad span state after sweepheapBitsSetTypeGCProg: small allocationmisrounded allocation in MHeap_SysAllocreceived on thread with no signal stackruntime: casfrom_Gscanstatus failed gp=runtime: typeBitsBulkBarrier with type stack growth not allowed in system calldeferproc: d.panic != nil after newdeferheapBitsBulkBarrier: unaligned argumentsreceived but handler not on signal stackrefill of span with free space remainingruntime.SetFinalizer: first argument is runtime: out of memory: cannot allocate value size not a multiple of value alignMHeap_FreeSpanLocked - invalid span stateMHeap_FreeSpanLocked - invalid stack freegcSweep being done but phase is not GCoffruntime.SetFinalizer: second argument is runtime.makemap: unsupported map key typeruntime: typeBitsBulkBarrier without typesetCheckmarked and isCheckmarked disagreevalue too large to be stored in data typeacquireSudog: found s.elem != nil in cachepersistentalloc: align is not a power of 2runtime: heapBitsSetTypeGCProg: total bitsunexpected signal during runtime executiongcBgMarkWorker: unexpected gcMarkWorkerModegentraceback before goexitPC initializationheapBitsSetTypeGCProg: unexpected bit countmultiple Read calls return no data or errorruntime.SetFinalizer: finalizer already setruntime.SetFinalizer: first argument is nilruntime: casfrom_Gscanstatus bad oldval gp=runtime: lfstackpush invalid packing: node=runtime: releaseSudog with non-nil gp.paramunknown runnable goroutine during bootstrapgcmarknewobject called while doing checkmarksystemstack called from unexpected goroutineexitsyscall: syscall frame is no longer validheapBitsSetType: called with non-pointer typeruntime: failed to create new OS thread (have runtime: name offset base pointer out of rangeruntime: panic before malloc heap initialized
runtime: text offset base pointer out of rangeruntime: type offset base pointer out of rangeP has cached GC work at end of mark terminationaddress family not supported by protocol familyP has unflushed stats at end of mark terminationruntime: cannot map pages in arena address spacewritebarrierptr_nostore1 called with mp.p == nilcasgstatus: waiting for Gwaiting but is Grunnableinvalid memory address or nil pointer dereferenceruntime: unexpected waitm - semaphore out of syncs.allocCount != s.nelems && freeIndex == s.nelemsmallocgc called with gcphase == _GCmarkterminationrecursive call during initialization - linker skewruntime: unable to acquire - semaphore out of syncruntime: cannot reserve arena virtual address spacecasfrom_Gscanstatus: gp->status is not in scan statefunction symbol table not sorted by program counter:runtime.SetFinalizer: pointer not in allocated blockruntime: use of FixAlloc_Alloc before FixAlloc_Init
runtime: failed to unwind through stackBarrier at SP 
goroutine running on other thread; stack unavailable
gcControllerState.findRunnable: blackening not enabledno goroutines (main called runtime.Goexit) - deadlock!casfrom_Gscanstatus:top gp->status is not in scan statenewproc: function arguments too large for new goroutinein gcMark expecting to see gcphase as _GCmarkterminationgentraceback cannot trace user goroutine on its own stacknon-Go code set up signal handler without SA_ONSTACK flagruntime:stoplockedm: g is not Grunnable or Gscanrunnable
found bad pointer in Go heap (incorrect use of unsafe or cgo?)runtime: internal error: misuse of lockOSThread/unlockOSThreadruntime: bsdthread_register error (unset DYLD_INSERT_LIBRARIES)runtime.SetFinalizer: pointer not at beginning of allocated blockruntime:greyobject: checkmarks finds unexpected unmarked object obj=runtime: found space for saved base pointer, but no framepointer experiment
bash-3.2$ 

The problem is due to swig ignoring the length of the go string and passing the p pointer directly down to bam.

bash-3.2$ find  $WORK -name main_wrap.cxx -exec grep -nA7 wrap_bam_main {} \;
246:void _wrap_bam_main_4e4163d4997ea833(_gostring_ _swig_go_0) {
247-  char *arg1 = (char *) (char *)0 ;
248-  
249-  arg1 = (char *)_swig_go_0.p; 
250-  
251-  bam((char const *)arg1);
252-  
253-}
bash-3.2$ 

Possible Fix

If we actually take into account the size of the string, we get the expected results both in interpreting the string literal as well as the argument passed in from the command line.

bash-3.2$ diff -cdp main_simple.i main_alloc_str.i 
*** main_simple.i	2016-12-16 22:51:50.000000000 -0800
--- main_alloc_str.i	2016-12-16 22:51:41.000000000 -0800
***************
*** 1,5 ****
--- 1,14 ----
  %module main
  %{
+ #include <string.h>
+ #include <stdlib.h>
  #include "bam.hpp"
  %}
+ %typemap(gotype) char *, char const *, char const * const %{string%}
+ %typemap(in) char *, char const *, char const * const {
+   $1 = strncpy((char *)malloc(sizeof(char) * ($input.n+1)), $input.p, $input.n);
+ }
+ %typemap(freearg) char *, char const *, char const * const {
+    free($1);
+ }
  %include "bam.hpp"
bash-3.2$ ln -fs main_alloc_str.i main.swigcxx
bash-3.2$ eval $(go build -work -o $EXE 2>&1) && $EXE 1st,2nd,3rd
1st
bash-3.2$ eval $(go build -work -o $EXE 2>&1) && $EXE
foo
bash-3.2$ 

That's because the wrappers used strncpy:

bash-3.2$ find  $WORK -name main_wrap.cxx -exec grep -nA12 wrap_bam_main {} \;
248:void _wrap_bam_main_d0a8e621b017f3f1(_gostring_ _swig_go_0) {
249-  char *arg1 = (char *) (char *)0 ;
250-  
251-  {
252-    arg1 = strncpy((char *)malloc(sizeof(char) * (_swig_go_0.n+1)), _swig_go_0.p, _swig_go_0.n);
253-  }
254-  
255-  bam((char const *)arg1);
256-  
257-  {
258-    free(arg1);
259-  }
260-}
bash-3.2$ 

A Better Solution

... probably involves inserting a call to cgo's C.CString and calling stdlib's free via C.free in the generated Go code instead of allocating and deallocating the string in the C wrapper.

[hzarnani]

Anybody taking on this bug? It's almost been two months since I reported it. And it's not a low-priority bug that can be put off indefinitely. It's pretty serious. And there's a solution as I described. It shouldn't take more than 10 minutes to add the code to Lib/go/std_string.i and test it.

[ojwb]

If you think you have the fix, submit a PR and it'll get automatically tested.

[tmc]

I believe this was fixed in 9cd3e28

[tmc]

This means that the 3.0.9 release fixed this.

[ojwb]

@hzarnani Please can you confirm whether this is fixed in 3.0.9 or newer?

[hzarnani]

I will check that ASAP (by tomorrow if not today). Thank you.

[tmc]

@hzarnani ping

[ojwb]

Looking at the code now generated for the example above, we now make a copy of the string using .p and .n, terminate it with a zero byte using .n and then pass the copy to the C code, so this is indeed fixed.