x/sys: Add Darwin support for clonefile and clonefileat

What version of Go are you using (`go version`)?

$ go version
1.14.2

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/stanhu/Library/Caches/go-build"
GOENV="/Users/stanhu/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/stanhu/.gvm/pkgsets/go1.14.2/global"
GOPRIVATE=""
GOPROXY="https://goproxy.io,direct"
GOROOT="/Users/stanhu/.gvm/gos/go1.14.2"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/stanhu/.gvm/gos/go1.14.2/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/stanhu/github/sys/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6v/0wvd8sg951l59wy4242m3thh0000gn/T/go-build411355479=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Attempted to build git-lfs with the latest Go version: git-lfs/git-lfs#4245

What did you expect to see?

No errors.

What did you see instead?

git-lfs has a dependency on direct system calls, which is no longer supported in Go v1.12:

util_darwin.go:127:3: undefined: unix.SYS_CLONEFILEAT

Is adding support a matter of just adding these lines?

diff --git a/unix/syscall_darwin.go b/unix/syscall_darwin.go
index e2a05ee..bd4b0a3 100644
--- a/unix/syscall_darwin.go
+++ b/unix/syscall_darwin.go
@@ -401,6 +401,8 @@ func Sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
 //sys	Chroot(path string) (err error)
 //sys	ClockGettime(clockid int32, time *Timespec) (err error)
 //sys	Close(fd int) (err error)
+//sys	Clonefile(path string, dst_dirfd int, buf []byte, flags int) (err error)
+//sys	Clonefileat(src_dirfd int, path string, dst_dirfd int, buf []byte, flags int) (err error)
 //sys	Dup(fd int) (nfd int, err error)
 //sys	Dup2(from int, to int) (err error)
 //sys	Exchangedata(path1 string, path2 string, options int) (err error)

Is adding support a matter of just adding these lines?

Yes, adding these lines and running mkall.sh (or at least go run mksyscall.go) for all supported GOARCHes should be enough. Would also be nice to have a test for these. Want to send a CL/PR?

@tklauser

@tklauser Sure, I will give this shot. I did notice on the latest master branch, running GOOS=darwin GOARCH=amd64 sh mkall.sh without changing anything resulted in this diff:

diff

diff --git a/unix/zerrors_darwin_amd64.go b/unix/zerrors_darwin_amd64.go
index e3ff2ee..60c4c7d 100644
--- a/unix/zerrors_darwin_amd64.go
+++ b/unix/zerrors_darwin_amd64.go
@@ -82,7 +82,7 @@ const (
 	ATTR_CMN_PAROBJID                 = 0x80
 	ATTR_CMN_RETURNED_ATTRS           = 0x80000000
 	ATTR_CMN_SCRIPT                   = 0x100
-	ATTR_CMN_SETMASK                  = 0x41c7ff00
+	ATTR_CMN_SETMASK                  = 0x51c7ff00
 	ATTR_CMN_USERACCESS               = 0x200000
 	ATTR_CMN_UUID                     = 0x800000
 	ATTR_CMN_VALIDMASK                = 0xffffffff
@@ -353,7 +353,7 @@ const (
 	DLT_LINUX_SLL                     = 0x71
 	DLT_LOOP                          = 0x6c
 	DLT_LTALK                         = 0x72
-	DLT_MATCHING_MAX                  = 0xf5
+	DLT_MATCHING_MAX                  = 0x10a
 	DLT_MATCHING_MIN                  = 0x68
 	DLT_MFR                           = 0xb6
 	DLT_MOST                          = 0xd3
@@ -394,6 +394,7 @@ const (
 	DLT_SYMANTEC_FIREWALL             = 0x63
 	DLT_TZSP                          = 0x80
 	DLT_USB                           = 0xba
+	DLT_USB_DARWIN                    = 0x10a
 	DLT_USB_LINUX                     = 0xbd
 	DLT_USB_LINUX_MMAPPED             = 0xdc
 	DLT_USER0                         = 0x93
@@ -438,8 +439,8 @@ const (
 	EVFILT_PROC                       = -0x5
 	EVFILT_READ                       = -0x1
 	EVFILT_SIGNAL                     = -0x6
-	EVFILT_SYSCOUNT                   = 0xf
-	EVFILT_THREADMARKER               = 0xf
+	EVFILT_SYSCOUNT                   = 0x11
+	EVFILT_THREADMARKER               = 0x11
 	EVFILT_TIMER                      = -0x7
 	EVFILT_USER                       = -0xa
 	EVFILT_VM                         = -0xc
@@ -501,6 +502,7 @@ const (
 	F_GETOWN                          = 0x5
 	F_GETPATH                         = 0x32
 	F_GETPATH_MTMINFO                 = 0x47
+	F_GETPATH_NOFIRMLINK              = 0x66
 	F_GETPROTECTIONCLASS              = 0x3f
 	F_GETPROTECTIONLEVEL              = 0x4d
 	F_GLOBAL_NOCACHE                  = 0x37
@@ -527,6 +529,7 @@ const (
 	F_SETPROTECTIONCLASS              = 0x40
 	F_SETSIZE                         = 0x2b
 	F_SINGLE_WRITER                   = 0x4c
+	F_SPECULATIVE_READ                = 0x65
 	F_THAW_FS                         = 0x36
 	F_TRANSCODEKEY                    = 0x4b
 	F_TRIM_ACTIVE_FILE                = 0x64
@@ -558,6 +561,7 @@ const (
 	IFF_UP                            = 0x1
 	IFNAMSIZ                          = 0x10
 	IFT_1822                          = 0x2
+	IFT_6LOWPAN                       = 0x40
 	IFT_AAL5                          = 0x31
 	IFT_ARCNET                        = 0x23
 	IFT_ARCNETPLUS                    = 0x24
@@ -771,7 +775,7 @@ const (
 	IPV6_FAITH                        = 0x1d
 	IPV6_FLOWINFO_MASK                = 0xffffff0f
 	IPV6_FLOWLABEL_MASK               = 0xffff0f00
-	IPV6_FLOW_ECN_MASK                = 0x300
+	IPV6_FLOW_ECN_MASK                = 0x3000
 	IPV6_FRAGTTL                      = 0x3c
 	IPV6_FW_ADD                       = 0x1e
 	IPV6_FW_DEL                       = 0x1f
@@ -900,6 +904,7 @@ const (
 	MADV_SEQUENTIAL                   = 0x2
 	MADV_WILLNEED                     = 0x3
 	MADV_ZERO_WIRED_PAGES             = 0x6
+	MAP_32BIT                         = 0x8000
 	MAP_ANON                          = 0x1000
 	MAP_ANONYMOUS                     = 0x1000
 	MAP_COPY                          = 0x2
@@ -916,6 +921,15 @@ const (
 	MAP_RESILIENT_CODESIGN            = 0x2000
 	MAP_RESILIENT_MEDIA               = 0x4000
 	MAP_SHARED                        = 0x1
+	MCAST_BLOCK_SOURCE                = 0x54
+	MCAST_EXCLUDE                     = 0x2
+	MCAST_INCLUDE                     = 0x1
+	MCAST_JOIN_GROUP                  = 0x50
+	MCAST_JOIN_SOURCE_GROUP           = 0x52
+	MCAST_LEAVE_GROUP                 = 0x51
+	MCAST_LEAVE_SOURCE_GROUP          = 0x53
+	MCAST_UNBLOCK_SOURCE              = 0x55
+	MCAST_UNDEFINED                   = 0x0
 	MCL_CURRENT                       = 0x1
 	MCL_FUTURE                        = 0x2
 	MNT_ASYNC                         = 0x40
@@ -927,6 +941,7 @@ const (
 	MNT_DOVOLFS                       = 0x8000
 	MNT_DWAIT                         = 0x4
 	MNT_EXPORTED                      = 0x100
+	MNT_EXT_ROOT_DATA_VOL             = 0x1
 	MNT_FORCE                         = 0x80000
 	MNT_IGNORE_OWNERSHIP              = 0x200000
 	MNT_JOURNALED                     = 0x800000
@@ -943,12 +958,15 @@ const (
 	MNT_QUOTA                         = 0x2000
 	MNT_RDONLY                        = 0x1
 	MNT_RELOAD                        = 0x40000
+	MNT_REMOVABLE                     = 0x200
 	MNT_ROOTFS                        = 0x4000
+	MNT_SNAPSHOT                      = 0x40000000
+	MNT_STRICTATIME                   = 0x80000000
 	MNT_SYNCHRONOUS                   = 0x2
 	MNT_UNION                         = 0x20
 	MNT_UNKNOWNPERMISSIONS            = 0x200000
 	MNT_UPDATE                        = 0x10000
-	MNT_VISFLAGMASK                   = 0x17f0f5ff
+	MNT_VISFLAGMASK                   = 0xd7f0f7ff
 	MNT_WAIT                          = 0x1
 	MSG_CTRUNC                        = 0x20
 	MSG_DONTROUTE                     = 0x4
@@ -975,9 +993,10 @@ const (
 	NET_RT_DUMP                       = 0x1
 	NET_RT_DUMP2                      = 0x7
 	NET_RT_FLAGS                      = 0x2
+	NET_RT_FLAGS_PRIV                 = 0xa
 	NET_RT_IFLIST                     = 0x3
 	NET_RT_IFLIST2                    = 0x6
-	NET_RT_MAXID                      = 0xa
+	NET_RT_MAXID                      = 0xb
 	NET_RT_STAT                       = 0x4
 	NET_RT_TRASH                      = 0x5
 	NFDBITS                           = 0x20
@@ -1015,6 +1034,7 @@ const (
 	NOTE_LEEWAY                       = 0x10
 	NOTE_LINK                         = 0x10
 	NOTE_LOWAT                        = 0x1
+	NOTE_MACHTIME                     = 0x100
 	NOTE_MACH_CONTINUOUS_TIME         = 0x80
 	NOTE_NONE                         = 0x80
 	NOTE_NSECONDS                     = 0x4
@@ -1132,6 +1152,7 @@ const (
 	RTF_BROADCAST                     = 0x400000
 	RTF_CLONING                       = 0x100
 	RTF_CONDEMNED                     = 0x2000000
+	RTF_DEAD                          = 0x20000000
 	RTF_DELCLONE                      = 0x80
 	RTF_DONE                          = 0x40
 	RTF_DYNAMIC                       = 0x10
@@ -1139,6 +1160,7 @@ const (
 	RTF_HOST                          = 0x4
 	RTF_IFREF                         = 0x4000000
 	RTF_IFSCOPE                       = 0x1000000
+	RTF_LLDATA                        = 0x400
 	RTF_LLINFO                        = 0x400
 	RTF_LOCAL                         = 0x200000
 	RTF_MODIFIED                      = 0x20
@@ -1206,6 +1228,7 @@ const (
 	SIOCGDRVSPEC                      = 0xc028697b
 	SIOCGETVLAN                       = 0xc020697f
 	SIOCGHIWAT                        = 0x40047301
+	SIOCGIF6LOWPAN                    = 0xc02069c5
 	SIOCGIFADDR                       = 0xc0206921
 	SIOCGIFALTMTU                     = 0xc0206948
 	SIOCGIFASYNCMAP                   = 0xc020697c
@@ -1216,6 +1239,7 @@ const (
 	SIOCGIFDEVMTU                     = 0xc0206944
 	SIOCGIFDSTADDR                    = 0xc0206922
 	SIOCGIFFLAGS                      = 0xc0206911
+	SIOCGIFFUNCTIONALTYPE             = 0xc02069ad
 	SIOCGIFGENERIC                    = 0xc020693a
 	SIOCGIFKPI                        = 0xc0206987
 	SIOCGIFMAC                        = 0xc0206982
@@ -1229,6 +1253,7 @@ const (
 	SIOCGIFSTATUS                     = 0xc331693d
 	SIOCGIFVLAN                       = 0xc020697f
 	SIOCGIFWAKEFLAGS                  = 0xc0206988
+	SIOCGIFXMEDIA                     = 0xc02c6948
 	SIOCGLOWAT                        = 0x40047303
 	SIOCGPGRP                         = 0x40047309
 	SIOCIFCREATE                      = 0xc0206978
@@ -1239,6 +1264,7 @@ const (
 	SIOCSDRVSPEC                      = 0x8028697b
 	SIOCSETVLAN                       = 0x8020697e
 	SIOCSHIWAT                        = 0x80047300
+	SIOCSIF6LOWPAN                    = 0x802069c4
 	SIOCSIFADDR                       = 0x8020690c
 	SIOCSIFALTMTU                     = 0x80206945
 	SIOCSIFASYNCMAP                   = 0x8020697d
diff --git a/unix/zsyscall_darwin_amd64.s b/unix/zsyscall_darwin_amd64.s
index a3fdf09..276514d 100644
--- a/unix/zsyscall_darwin_amd64.s
+++ b/unix/zsyscall_darwin_amd64.s
@@ -164,6 +164,8 @@ TEXT ·libc_getrusage_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_getrusage(SB)
 TEXT ·libc_getsid_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_getsid(SB)
+TEXT ·libc_gettimeofday_trampoline(SB),NOSPLIT,$0-0
+	JMP	libc_gettimeofday(SB)
 TEXT ·libc_getuid_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_getuid(SB)
 TEXT ·libc_issetugid_trampoline(SB),NOSPLIT,$0-0
@@ -266,8 +268,6 @@ TEXT ·libc_munmap_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_munmap(SB)
 TEXT ·libc_ptrace_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_ptrace(SB)
-TEXT ·libc_gettimeofday_trampoline(SB),NOSPLIT,$0-0
-	JMP	libc_gettimeofday(SB)
 TEXT ·libc_fstat64_trampoline(SB),NOSPLIT,$0-0
 	JMP	libc_fstat64(SB)
 TEXT ·libc_fstatat64_trampoline(SB),NOSPLIT,$0-0
diff --git a/unix/ztypes_darwin_amd64.go b/unix/ztypes_darwin_amd64.go
index 966798a..06d637e 100644
--- a/unix/ztypes_darwin_amd64.go
+++ b/unix/ztypes_darwin_amd64.go
@@ -70,7 +70,6 @@ type Stat_t struct {
 	Uid     uint32
 	Gid     uint32
 	Rdev    int32
-	_       [4]byte
 	Atim    Timespec
 	Mtim    Timespec
 	Ctim    Timespec
@@ -97,10 +96,11 @@ type Statfs_t struct {
 	Type        uint32
 	Flags       uint32
 	Fssubtype   uint32
-	Fstypename  [16]int8
-	Mntonname   [1024]int8
-	Mntfromname [1024]int8
-	Reserved    [8]uint32
+	Fstypename  [16]byte
+	Mntonname   [1024]byte
+	Mntfromname [1024]byte
+	Flags_ext   uint32
+	Reserved    [7]uint32
 }
 
 type Flock_t struct {
@@ -133,8 +133,7 @@ type Fbootstraptransfer_t struct {
 
 type Log2phys_t struct {
 	Flags uint32
-	_     [8]byte
-	_     [8]byte
+	_     [16]byte
 }
 
 type Fsid struct {
@@ -221,10 +220,8 @@ type IPv6Mreq struct {
 type Msghdr struct {
 	Name       *byte
 	Namelen    uint32
-	_          [4]byte
 	Iov        *Iovec
 	Iovlen     int32
-	_          [4]byte
 	Control    *byte
 	Controllen uint32
 	Flags      int32
@@ -309,7 +306,6 @@ type IfMsghdr struct {
 	Addrs   int32
 	Flags   int32
 	Index   uint16
-	_       [2]byte
 	Data    IfData
 }
 
@@ -352,7 +348,6 @@ type IfaMsghdr struct {
 	Addrs   int32
 	Flags   int32
 	Index   uint16
-	_       [2]byte
 	Metric  int32
 }
 
@@ -373,7 +368,6 @@ type IfmaMsghdr2 struct {
 	Addrs    int32
 	Flags    int32
 	Index    uint16
-	_        [2]byte
 	Refcount int32
 }
 
@@ -382,7 +376,6 @@ type RtMsghdr struct {
 	Version uint8
 	Type    uint8
 	Index   uint16
-	_       [2]byte
 	Flags   int32
 	Addrs   int32
 	Pid     int32
@@ -404,7 +397,8 @@ type RtMetrics struct {
 	Rtt      uint32
 	Rttvar   uint32
 	Pksent   uint32
-	Filler   [4]uint32
+	State    uint32
+	Filler   [3]uint32
 }
 
 const (
@@ -427,7 +421,6 @@ type BpfStat struct {
 
 type BpfProgram struct {
 	Len   uint32
-	_     [4]byte
 	Insns *BpfInsn
 }
 
@@ -452,7 +445,6 @@ type Termios struct {
 	Cflag  uint64
 	Lflag  uint64
 	Cc     [20]uint8
-	_      [4]byte
 	Ispeed uint64
 	Ospeed uint64
 }

Is this expected?

@tklauser

@tklauser Sure, I will give this shot. I did notice on the latest master branch, running GOOS=darwin GOARCH=amd64 sh mkall.sh without changing anything resulted in this diff:
diff

Is this expected?

@stanhu Yes, this is expected. mkall.sh was probably run on an older macOS versions and some additional consts/types have been added in the meantime. For the matter of this change you can ignore these parts of the diff.

unix: add Darwin support for clonefile syscalls

@tklauser

@tklauser Thanks, I created https://go-review.googlesource.com/c/sys/+/254757.

Change https://golang.org/cl/254757 mentions this issue: unix: add Darwin support for clonefile syscalls

util_darwin.go: Remove use of direct syscalls

tools/util_darwin.go: Remove use of direct syscalls

unix: add Darwin support for clonefile syscalls

Navigation Menu

Repository navigation

x/sys: Add Darwin support for clonefile and clonefileat #41366

What version of Go are you using (`go version`)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (`go env`)?

What did you do?

What did you expect to see?

What did you see instead?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

x/sys: Add Darwin support for clonefile and clonefileat #41366

Description

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

Activity

tklauser commented on Sep 14, 2020

stanhu commented on Sep 14, 2020

tklauser commented on Sep 14, 2020

stanhu commented on Sep 14, 2020

gopherbot commented on Sep 15, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?