x/mobile: fatal error: bulkBarrierPreWrite: unaligned arguments #46893

champo · 2021-06-23T17:11:52Z

What version of Go are you using (`go version`)?

$ go version
go version go1.16.5 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/juancivile/Library/Caches/go-build"
GOENV="/Users/juancivile/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/juancivile/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.16.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.16.5/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.5"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ln/73tvl7md2l7gqpbkzq439dcc0000gn/T/go-build3867799831=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have an iOS project using a go library via gomobile:

Running on an arm64 device (any modern iPhone)
Calling a method that takes and returns []byte
A ton of invocations. This is affecting randomly about 1-2% of users of our app.

The golang code is simple:

type Panicker interface {
	Panic([]byte) ([]byte, error)
}

type PanickerImpl struct {
}

func (p *PanickerImpl) Panic(b []byte) ([]byte, error) {
	return make([]byte, len(b)), nil
}

func Init() {
	debug.SetTraceback("crash")
}

And on swift:

PanickerInit()

let p = PanickerPanickerImpl()
let d = Data(repeating: 1, count: 10)
for _ in 1...10_000_000 {
      let r = try p.panic(d)
      XCTAssertEqual(r.count, d.count)
}

A full working reproduction project is at https://github.com/champo/gomobile_panic

What did you expect to see?

No crash.

What did you see instead?

It sometimes crashes inside bulkBarrierPreWrite:

fatal error: bulkBarrierPreWrite: unaligned arguments

goroutine 17 [running, locked to thread]:
runtime.throw(0x1014aeaa4, 0x28)
	runtime/panic.go:1117 +0x54 fp=0x130582d30 sp=0x130582d00 pc=0x1014373d4
runtime.bulkBarrierPreWrite(0x16f5e483c, 0x130582e68, 0x8)
	runtime/mbitmap.go:554 +0x44c fp=0x130582dd0 sp=0x130582d30 pc=0x101419f0c
runtime.typedmemmove(0x1014e0000, 0x16f5e483c, 0x130582e68)
	runtime/mbarrier.go:161 +0xa4 fp=0x130582e10 sp=0x130582dd0 pc=0x1014190c4
_cgoexp_59f89a9c1f1d_proxypanicker_PanickerImpl_Panic(0x16f5e4824)
	_cgo_gotypes.go:183 +0xe8 fp=0x130582e80 sp=0x130582e10 pc=0x1014a3068
runtime.cgocallbackg1(0x1014a2f80, 0x16f5e4824, 0x0)
	runtime/cgocall.go:292 +0x140 fp=0x130582f40 sp=0x130582e80 pc=0x10140a9e0
runtime.cgocallbackg(0x1014a2f80, 0x16f5e4824, 0x0)
	runtime/cgocall.go:228 +0xb0 fp=0x130582fb0 sp=0x130582f40 pc=0x10140a810
runtime.cgocallback(0x0, 0x0, 0x0)
	runtime/asm_arm64.s:1055 +0x98 fp=0x130582fe0 sp=0x130582fb0 pc=0x1014687f8
runtime.goexit()
	runtime/asm_arm64.s:1130 +0x4 fp=0x130582fe0 sp=0x130582fe0 pc=0x1014688d4

I tried exposing a free function with a similar signature but no crashers there. Removing the paramters or return values seems to avoid the crash too.

The text was updated successfully, but these errors were encountered:

rayvbr · 2021-06-25T18:19:24Z

I've encountered this problem as well in the past, issue started with Go 1.16 (1.15 seems unaffected). In my case I had a function signature func DoSomething(i int32, s string) (string, error). Changing that to func DoSomething(s string, i int32) (string, error) resolved the issue. I never really understood why

hilariocoelho · 2021-06-28T16:31:25Z

I had the same issue using Go 1.16. Reverted to Go 1.13 and it works fine. Also just affected a few users and couldn't reproduce it consistently.

marinthiercelin · 2021-07-21T15:51:40Z

I have experienced the same issue using https://github.com/ProtonMail/gopenpgp in an iOS app (with gomobile).
The issue has been reproduced when building the library with golang 1.16, 1.16.5 and 1.16.6.
Building with golang 1.15.14 works fine.

elagergren-spideroak · 2021-12-17T08:53:04Z

Can consistently repro with 1.17.x on iOS. It occurs when calling into Go from C. The stack frame C sets up for Go is four-byte aligned, which makes it likely that the address of the results aren't properly aligned, and therefore casues typedmemmove to crash. For exampe: _cgoexp_432d4433717a_proxygolib_Client_Query(0x16f8557d4).

I think this is a cmd/cgo problem, not x/mobile. cc: @ianlancetaylor

elagergren-spideroak · 2022-01-03T18:22:05Z

An update: we patched cmd/cgo with __attribute__((aligned(8))) and it fixes the problem.

hilariocoelho · 2022-01-03T22:00:30Z

@elagergren-spideroak when should we expect a release that fixes it, do you have any idea?

elagergren-spideroak · 2022-01-04T17:31:28Z

I am not a member of the Go team, so I am not the person to ask. However, I'm going to submit a CL with the patch to get things started.

sanderdekoning · 2022-01-16T13:53:40Z

Bump - just ran into this: fatal error: bulkBarrierPreWrite: unaligned arguments

sanderdekoning · 2022-01-18T18:03:20Z

@elagergren-spideroak just to know if I/we can assist in any way, the current status is what exactly? Thanks for letting us know how you fixed it!

champo · 2022-01-24T16:30:27Z

@elagergren-spideroak thanks for looking into this! Did you get a chance to make the CL? If you didn't, I'd really appreciate a gist so I can patch locally for the time being. I just got a new batch of ocurrences of this I can't seem to sidestep :(

elagergren-spideroak · 2022-01-24T18:04:01Z

How we've fixed it for our iOS builds is by replacing

go/src/cmd/cgo/out.go

Line 1005 in 97e740e

    
           fmt.Fprintf(fgcc, "\ttypedef %s %v _cgo_argtype;\n", ctype, p.packedAttribute())

with

fmt.Fprintf(fgcc, "\ttypedef %s %v __attribute__((aligned(8))) _cgo_argtype;\n", ctype, p.packedAttribute())

champo · 2022-01-27T20:35:42Z

Thanks @elagergren-spideroak! Just tested it out and works great 👌

alock · 2022-02-28T18:21:48Z

I am not a member of the Go team, so I am not the person to ask. However, I'm going to submit a CL with the patch to get things started.

@elagergren-spideroak Where you able to create a CL? If so can you share the link to follow here.

Irregularly typedmemmove and bulkBarrierPreWrite crashes on unaligned arguments. By aligning the arguments on 8 bytes this is fixed. Fixes golang#46893

gopherbot · 2022-05-25T07:37:03Z

Change https://go.dev/cl/408395 mentions this issue: cmd/cgo: fix unaligned arguments typedmemmove crash on iOS

lixin9311 · 2022-07-07T02:17:39Z

Thanks for the fix. It fixed our runtime.raise_trampoline crashed.

junyang100 · 2022-07-08T03:13:34Z

I just started using Golang. Can anyone tell me if I need to recompile the Golang or just need to rebuild my project after I change the source code

jeffrrogers · 2022-12-16T19:08:36Z

@gopherbot
This still seems to be an issue when running on ARM architecture (MacOS M1 or iOS devices). We compiled our framework with Go version 1.19.4 on both Intel and M1 machines, and it runs fine on Intel machine simulators, but crashes intermittently on M1 machine simulators and iOS devices. This was supposed to be fixed in 1.19.2 right? Here is the stack trace we get for our crash:

fatal error: bulkBarrierPreWrite: unaligned arguments

goroutine 17 [running, locked to thread]:
runtime.throw({0x1013583ac?, 0x1300ddd08?})
/usr/local/go/src/runtime/panic.go:1047 +0x40 fp=0x1300ddce0 sp=0x1300ddcb0 pc=0x101095a90
runtime.bulkBarrierPreWrite(0x1300ddd78?, 0x0?, 0x130543530?)
/usr/local/go/src/runtime/mbitmap.go:582 +0x420 fp=0x1300ddd50 sp=0x1300ddce0 pc=0x1010759c0
runtime.typedmemmove(0x1014ba460, 0x16f4eb6bc, 0x1300ddde8)
/usr/local/go/src/runtime/mbarrier.go:162 +0x48 fp=0x1300ddd90 sp=0x1300ddd50 pc=0x101074bd8
_cgoexp_409b037657a8_proxyswiftgold_SwiftGoLD_Normalize(0x16f4eb6a4)
_cgo_gotypes.go:168 +0xa4 fp=0x1300dde00 sp=0x1300ddd90 pc=0x1012900d4
runtime.cgocallbackg1(0x101290030, 0x1300ddfe0?, 0x0)
/usr/local/go/src/runtime/cgocall.go:316 +0x244 fp=0x1300ddef0 sp=0x1300dde00 pc=0x1010653a4
runtime.cgocallbackg(0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/cgocall.go:235 +0xd8 fp=0x1300ddf80 sp=0x1300ddef0 pc=0x1010650d8
runtime.cgocallbackg(0x101290030, 0x16f4eb6a4, 0x0)
:1 +0x1c fp=0x1300ddfb0 sp=0x1300ddf80 pc=0x1010c694c
runtime.cgocallback(0x0, 0x0, 0x0)
/usr/local/go/src/runtime/asm_arm64.s:1094 +0xa0 fp=0x1300ddfe0 sp=0x1300ddfb0 pc=0x1010c52b0
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1300ddfe0 sp=0x1300ddfe0 pc=0x1010c5384

goroutine 21 [GC worker (idle)]:
runtime.gopark(0x18ff8289ace30?, 0x2?, 0x70?, 0x31?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x13003cf40 sp=0x13003cf20 pc=0x1010983d4
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1235 +0xec fp=0x13003cfd0 sp=0x13003cf40 pc=0x10107c20c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x13003cfd0 sp=0x13003cfd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x13003ffa0 sp=0x13003ff80 pc=0x1010983d4
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:369
runtime.forcegchelper()
/usr/local/go/src/runtime/proc.go:302 +0xac fp=0x13003ffd0 sp=0x13003ffa0 pc=0x10109826c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x13003ffd0 sp=0x13003ffd0 pc=0x1010c5384
created by runtime.init.6
/usr/local/go/src/runtime/proc.go:290 +0x24

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x130040f70 sp=0x130040f50 pc=0x1010983d4
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:369
runtime.bgsweep(0x0?)
/usr/local/go/src/runtime/mgcsweep.go:297 +0x10c fp=0x130040fb0 sp=0x130040f70 pc=0x10108586c
runtime.gcenable.func1()
/usr/local/go/src/runtime/mgc.go:178 +0x28 fp=0x130040fd0 sp=0x130040fb0 pc=0x10107a0a8
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x130040fd0 sp=0x130040fd0 pc=0x1010c5384
created by runtime.gcenable
/usr/local/go/src/runtime/mgc.go:178 +0x74

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x1f4dfcc?, 0xbf69ca?, 0x0?, 0x0?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x130041f50 sp=0x130041f30 pc=0x1010983d4
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:369
runtime.(*scavengerState).park(0x10171be40)
/usr/local/go/src/runtime/mgcscavenge.go:389 +0x5c fp=0x130041f80 sp=0x130041f50 pc=0x1010837bc
runtime.bgscavenge(0x0?)
/usr/local/go/src/runtime/mgcscavenge.go:622 +0xac fp=0x130041fb0 sp=0x130041f80 pc=0x101083dcc
runtime.gcenable.func2()
/usr/local/go/src/runtime/mgc.go:179 +0x28 fp=0x130041fd0 sp=0x130041fb0 pc=0x10107a048
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x130041fd0 sp=0x130041fd0 pc=0x1010c5384
created by runtime.gcenable
/usr/local/go/src/runtime/mgc.go:179 +0xb8

goroutine 18 [finalizer wait]:
runtime.gopark(0x10171c6e0?, 0x130082340?, 0x0?, 0x0?, 0x1?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x13003ed80 sp=0x13003ed60 pc=0x1010983d4
runtime.goparkunlock(...)
/usr/local/go/src/runtime/proc.go:369
runtime.runfinq()
/usr/local/go/src/runtime/mfinal.go:180 +0x124 fp=0x13003efd0 sp=0x13003ed80 pc=0x1010792d4
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x13003efd0 sp=0x13003efd0 pc=0x1010c5384
created by runtime.createfing
/usr/local/go/src/runtime/mfinal.go:157 +0x84

goroutine 19 [select, locked to thread]:
runtime.gopark(0x13003afa0?, 0x2?, 0x88?, 0xae?, 0x13003af9c?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x13003ae30 sp=0x13003ae10 pc=0x1010983d4
runtime.selectgo(0x13003afa0, 0x13003af98, 0x0?, 0x0, 0x0?, 0x1)
/usr/local/go/src/runtime/select.go:328 +0x68c fp=0x13003af50 sp=0x13003ae30 pc=0x1010a821c
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal_unix.go:991 +0x190 fp=0x13003afd0 sp=0x13003af50 pc=0x1010ac2c0
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x13003afd0 sp=0x13003afd0 pc=0x1010c5384
created by runtime.ensureSigM
/usr/local/go/src/runtime/signal_unix.go:974 +0xf4

goroutine 20 [syscall]:
runtime.sigNoteSleep(0x0)
/usr/local/go/src/runtime/os_darwin.go:123 +0x20 fp=0x13003bf90 sp=0x13003bf50 pc=0x101092c70
os/signal.signal_recv()
/usr/local/go/src/runtime/sigqueue.go:149 +0x2c fp=0x13003bfb0 sp=0x13003bf90 pc=0x1010c183c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x1c fp=0x13003bfd0 sp=0x13003bfb0 pc=0x10128f74c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x13003bfd0 sp=0x13003bfd0 pc=0x1010c5384
created by os/signal.Notify.func1.1
/usr/local/go/src/os/signal/signal.go:151 +0x2c

goroutine 35 [runnable]:
runtime.gcMarkDone()
/usr/local/go/src/runtime/mgc.go:784 +0x304 fp=0x1305e6f40 sp=0x1305e6f40 pc=0x10107b044
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1367 +0x3c0 fp=0x1305e6fd0 sp=0x1305e6f40 pc=0x10107c4e0
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1305e6fd0 sp=0x1305e6fd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

goroutine 36 [GC worker (idle)]:
runtime.gopark(0x18ff819afa669?, 0x13045c800?, 0x18?, 0x14?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x1305e7f40 sp=0x1305e7f20 pc=0x1010983d4
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1235 +0xec fp=0x1305e7fd0 sp=0x1305e7f40 pc=0x10107c20c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1305e7fd0 sp=0x1305e7fd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

goroutine 5 [GC worker (idle)]:
runtime.gopark(0x18ff7e15e0483?, 0x2?, 0x44?, 0x4f?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x1305e2f40 sp=0x1305e2f20 pc=0x1010983d4
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1235 +0xec fp=0x1305e2fd0 sp=0x1305e2f40 pc=0x10107c20c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1305e2fd0 sp=0x1305e2fd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

goroutine 37 [GC worker (idle)]:
runtime.gopark(0x18ff82e592446?, 0x2?, 0x5e?, 0xa0?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x1305e8f40 sp=0x1305e8f20 pc=0x1010983d4
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1235 +0xec fp=0x1305e8fd0 sp=0x1305e8f40 pc=0x10107c20c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1305e8fd0 sp=0x1305e8fd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

goroutine 38 [GC worker (idle)]:
runtime.gopark(0x18ff82e55881f?, 0x1?, 0x84?, 0x8f?, 0x0?)
/usr/local/go/src/runtime/proc.go:363 +0xe4 fp=0x1305e9f40 sp=0x1305e9f20 pc=0x1010983d4
runtime.gcBgMarkWorker()
/usr/local/go/src/runtime/mgc.go:1235 +0xec fp=0x1305e9fd0 sp=0x1305e9f40 pc=0x10107c20c
runtime.goexit()
/usr/local/go/src/runtime/asm_arm64.s:1172 +0x4 fp=0x1305e9fd0 sp=0x1305e9fd0 pc=0x1010c5384
created by runtime.gcBgMarkStartWorkers
/usr/local/go/src/runtime/mgc.go:1159 +0x28

randall77 · 2022-12-16T21:36:24Z

I don't think this has actually been fixed yet. There is a CL (https://go-review.googlesource.com/c/go/+/408395) but it hasn't been checked in yet.
#55122 has been fixed and backported, but I don't think that's the same issue as this one. This one has to do with cgo.

gopherbot added the mobile Android, iOS, and x/mobile label Jun 23, 2021

gopherbot added this to the Unreleased milestone Jun 23, 2021

toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 24, 2021

timcooijmans linked a pull request May 25, 2022 that will close this issue

cmd/cgo: fix unaligned arguments typedmemmove crash on iOS #53064

Open

Roasbeef mentioned this issue Sep 17, 2022

fatal error: bulkBarrierPreWrite: unaligned arguments (go 1.19.1) lightningnetwork/lnd#6923

Closed

x/mobile: fatal error: bulkBarrierPreWrite: unaligned arguments #46893

x/mobile: fatal error: bulkBarrierPreWrite: unaligned arguments #46893

champo commented Jun 23, 2021

rayvbr commented Jun 25, 2021

hilariocoelho commented Jun 28, 2021

marinthiercelin commented Jul 21, 2021

elagergren-spideroak commented Dec 17, 2021 •

edited

elagergren-spideroak commented Jan 3, 2022 •

edited

hilariocoelho commented Jan 3, 2022

elagergren-spideroak commented Jan 4, 2022

sanderdekoning commented Jan 16, 2022

sanderdekoning commented Jan 18, 2022

champo commented Jan 24, 2022

elagergren-spideroak commented Jan 24, 2022

champo commented Jan 27, 2022

alock commented Feb 28, 2022

gopherbot commented May 25, 2022

lixin9311 commented Jul 7, 2022

junyang100 commented Jul 8, 2022

jeffrrogers commented Dec 16, 2022

randall77 commented Dec 16, 2022

x/mobile: fatal error: bulkBarrierPreWrite: unaligned arguments #46893

x/mobile: fatal error: bulkBarrierPreWrite: unaligned arguments #46893

Comments

champo commented Jun 23, 2021

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

rayvbr commented Jun 25, 2021

hilariocoelho commented Jun 28, 2021

marinthiercelin commented Jul 21, 2021

elagergren-spideroak commented Dec 17, 2021 • edited

elagergren-spideroak commented Jan 3, 2022 • edited

hilariocoelho commented Jan 3, 2022

elagergren-spideroak commented Jan 4, 2022

sanderdekoning commented Jan 16, 2022

sanderdekoning commented Jan 18, 2022

champo commented Jan 24, 2022

elagergren-spideroak commented Jan 24, 2022

champo commented Jan 27, 2022

alock commented Feb 28, 2022

gopherbot commented May 25, 2022

lixin9311 commented Jul 7, 2022

junyang100 commented Jul 8, 2022

jeffrrogers commented Dec 16, 2022

randall77 commented Dec 16, 2022

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

elagergren-spideroak commented Dec 17, 2021 •

edited

elagergren-spideroak commented Jan 3, 2022 •

edited