Unable To Append Two Emoji Strings On macOS Ventura's Swift #63664

brianmichel · 2023-02-14T23:55:17Z

Description
Appending two emoji strings together crashes on macOS Ventura

Steps to reproduce

Use macOS Ventura
Run the sample package's tests or
try to execute the code "⛓".appending("🦑") in the Swift repl

Expected behavior
This code should run and execute fine producing a string that is equal to "⛓🦑"

Actual behavior
This works correctly on macOS Monterey, but crashes on macOS Ventura with the message Swift/StringUTF16View.swift:144: Fatal error: String index is out of bounds

Environment

Swift compiler version info
-- swift-driver version: 1.62.15 Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
-- Target: arm64-apple-macosx13.0
Xcode version info
-- Xcode 14.2
-- Build version 14C18
Deployment target: macOS 11

Sample Project
FuzzyMatchingPackage.zip

The text was updated successfully, but these errors were encountered:

WowbaggersLiquidLunch · 2023-02-15T00:34:31Z

I'm not able to reproduce the error. Could there be some bugs in your appending function? The standard library has only append functions.

amonshiz · 2023-02-15T00:39:09Z

appending is in the StringProtocol. https://developer.apple.com/documentation/swift/stringprotocol/appending(_:)

danielpunkass · 2023-02-15T00:41:45Z

FWIW I also am not able to reproduce the crash on Xcode Version 14.2 (14C18). Ventura 13.2.1 (22D68). (swiftlang-5.7.2.135.5 clang-1400.0.29.51)

brianmichel · 2023-02-15T01:29:34Z

I'm going to close this for now since it feels like we're missing a step somewhere in the repro (or some specific machine configuration).

AnthonyLatsis · 2023-02-15T11:38:58Z

appending is in the StringProtocol.

Yes, but regrettably the documentation does not mention that the function is declared in Foundation, not the Standard Library.

@brianmichel Does "⛓".appending("🦑") crash in the REPL by itself, or after importing Foundation?

fischman-bcny · 2023-02-15T17:36:27Z

@AnthonyLatsis Foundation needs to be imported first. This is what I see (note use of \u{} to make input 7-bit clean, and env -i usage to rule out locale-related env vars' impact):

~ $ env -i TERM=vt100 PATH=/usr/bin /usr/bin/swift repl
Welcome to Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51).
Type :help for assistance.
  1> "\u{26D3}".appending("\u{1F991}")
expression failed to parse:
error: repl.swift:1:12: error: value of type 'String' has no member 'appending'
"\u{26D3}".appending("\u{1F991}")
~~~~~~~~~~ ^~~~~~~~~


  1> import Foundation
  2> "\u{26D3}".appending("\u{1F991}")
Swift/StringUTF16View.swift:144: Fatal error: String index is out of bounds
2023-02-15 09:34:51.301761-0800 repl_swift[17376:2992790] Swift/StringUTF16View.swift:144: Fatal error: String index is out of bounds
Execution interrupted. Enter code to recover and continue.
Enter LLDB commands to investigate (type :help for assistance.)

Note that adding an explicit NSString() to the mix makes the crash go away:

  3> NSString("\u{26D3}").appending("\u{1F991}")
$R0: String = "⛓🦑"

(this is on Ventura 13.2.1 (22D68) on a 2021 M1 Pro)

danielpunkass · 2023-02-15T19:01:02Z

When I follow precisely the same steps as @fischman-bcny, on the same OS, also with an M1 (Max) MacBook Pro, I don't see the same crash. Could we be getting different Foundations?

ProductName:		macOS
ProductVersion:		13.2.1
BuildVersion:		22D68
daniel@Taco ~/Sources/RSFoundation % env -i TERM=vt100 PATH=/usr/bin /usr/bin/swift repl
Welcome to Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51).
Type :help for assistance.
  1>  "\u{26D3}".appending("\u{1F991}")
expression failed to parse:
error: repl.swift:1:13: error: value of type 'String' has no member 'appending'
 "\u{26D3}".appending("\u{1F991}")
 ~~~~~~~~~~ ^~~~~~~~~


  1> import Foundation
  2> "\u{26D3}".appending("\u{1F991}")
$R0: String = "⛓🦑"
  3>

AnthonyLatsis · 2023-02-15T19:03:53Z

Thanks for clarifying. One more thing: Does it crash if you interpret it with swift file.swift?

// file.swift
import Foundation
let _ = "\u{26D3}".appending("\u{1F991}")

AnthonyLatsis · 2023-02-15T19:21:00Z

Could we be getting different Foundations?

Not if you both have a single Xcode installed and their versions match, but you can check for certain with print(NSFoundationVersionNumber). Maybe the crash is non-deterministic?

fischman-bcny · 2023-02-15T19:39:27Z

@AnthonyLatsis this is what I see:

$ cat file.swift ; echo; env -i PATH=/usr/bin /usr/bin/swift file.swift
import Foundation
print(NSFoundationVersionNumber)
let _ = "\u{26D3}".appending("\u{1F991}")

1953.3
Swift/StringUTF16View.swift:144: Fatal error: String index is out of bounds
Stack dump:
0.      Program arguments: /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-frontend -frontend -interpret file.swift -Xllvm -aarch64-use-tbi -enable-objc-interop -stack-check -sdk /Applications/Xcode-14.2.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -color-diagnostics -new-driver-path /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-driver -empty-abi-descriptor -resource-dir /Applications/Xcode-14.2.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift -module-name file -disable-clang-spi -target-sdk-version 13.1
1.      Apple Swift version 5.7.2 (swiftlang-5.7.2.135.5 clang-1400.0.29.51)
2.      Compiling with the current language version
3.      While running user code "file.swift"
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  swift-frontend           0x0000000104bb35b0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1  swift-frontend           0x0000000104bb25b4 llvm::sys::RunSignalHandlers() + 112
2  swift-frontend           0x0000000104bb3c34 SignalHandler(int) + 344
3  libsystem_platform.dylib 0x0000000196c702a4 _sigtramp + 56
4  libswiftCore.dylib       0x00000001a4a20fa0 $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_A2HSus6UInt32VtFySRys5UInt8VGXEfU_yAMXEfU_yAMXEfU_ + 380
5  libswiftCore.dylib       0x00000001a4a20ce4 $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_A2HSus6UInt32VtFySRys5UInt8VGXEfU_yAMXEfU_ + 200
6  libswiftCore.dylib       0x00000001a4a20adc $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_A2HSus6UInt32VtFySRys5UInt8VGXEfU_ + 212
7  libswiftCore.dylib       0x00000001a4a2063c $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_A2HSus6UInt32VtF + 236
8  libswiftCore.dylib       0x00000001a4a253b0 $sSKsE6_index_8offsetBy5IndexQzAD_SitFSS9UTF16ViewV_Tg5 + 1248
9  libswiftCore.dylib       0x00000001a4c0a084 $sSS9UTF16ViewV15_nativeGetIndex3forSS0E0VSi_tF + 272
10 libswiftCore.dylib       0x00000001a4c05860 $ss15__StringStorageC13getCharacters_5rangeySpys6UInt16VG_So13_SwiftNSRangeatF + 128
11 libswiftCore.dylib       0x00000001a4c059d4 $ss15__StringStorageC13getCharacters_5rangeySpys6UInt16VG_So13_SwiftNSRangeatFTo + 36
12 CoreFoundation           0x0000000196cca740 __CFStringEncodeByteStream + 2708
13 Foundation               0x0000000197bd52d4 -[NSString(NSStringOtherEncodings) getBytes:maxLength:usedLength:encoding:options:range:remainingRange:] + 260
14 Foundation               0x0000000197bda0cc _NSNewStringByAppendingStrings + 440
15 Foundation               0x0000000197bdab14 -[NSString stringByAppendingString:] + 76
16 Foundation               0x0000000197ff921c $sSy10FoundationE9appendingySSqd__SyRd__lF + 148
17 Foundation               0x000000010a2f0118 $sSy10FoundationE9appendingySSqd__SyRd__lF + 18446744071330295696
18 swift-frontend           0x000000010116b998 llvm::orc::runAsMain(int (*)(int, char**), llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, llvm::Optional<llvm::StringRef>) + 1312
19 swift-frontend           0x0000000100449250 swift::RunImmediately(swift::CompilerInstance&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, swift::IRGenOptions const&, swift::SILOptions const&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >&&) + 11268
20 swift-frontend           0x00000001001feccc performCompileStepsPostSILGen(swift::CompilerInstance&, std::__1::unique_ptr<swift::SILModule, std::__1::default_delete<swift::SILModule> >, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) + 2856
21 swift-frontend           0x0000000100200ce8 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 7284
22 swift-frontend           0x00000001001a1294 swift::mainEntry(int, char const**) + 3940
23 dyld                     0x0000000196917e50 start + 2544
Trace/BPT trap: 5

AnthonyLatsis · 2023-02-15T20:03:26Z

cc @parkera

fischman-bcny · 2023-03-20T16:06:03Z

Ping :)
Am interested in any other suggestions that might help divine where my system is differing from others' on this thread (and whether that difference matters to the software we ship)

compnerd · 2023-05-01T17:04:56Z

CC: @lorentey - in case something about the bridging stands out to him

lorentey · 2023-05-02T01:03:05Z

StringProtocol.appending is going out of its way to bridge its input as many times as possible, for no good reason. It's long overdue for a rewrite (and/or deprecation), but it should never crash.

The crash may be due to a bug in one of these related String changes:

#62717
#63592
#63646

I'll take a closer look.

lorentey · 2023-05-02T01:15:10Z

One very reliable workaround is to avoid calling StringProtocol.appending, preferring to use the Swift native + operator instead. (There are no drawbacks to doing that. Besides investigating the potential stdlib issue, I'll also try to change the implementation of appending to do the same. However, that change will not apply retroactively, even if it ends up shipping. + does the right thing on all platforms.)

lorentey · 2023-05-02T02:47:24Z

This is reliably reproducible by overriding Core Foundation's default text encoding from Mac OS Roman to UTF-8:

$ cat foo.swift
import Foundation
let _ = "\u{26D3}".appending("\u{1F991}")
$ swiftc foo.swift
$ __CF_USER_TEXT_ENCODING="$(id -u):0x08000100:0" ./foo
Swift/StringUTF16View.swift:147: Fatal error: String index is out of bounds
Trace/BPT trap: 5

This switches concatenation to a different algorithm, which exhibits the issue. I'm continuing to investigate.

(UTF-8 is likely to be the default text encoding at least in some regions, but evidently not all.)

fischman-bcny · 2023-05-02T19:48:05Z

@lorentey the tip to use + worked a treat for the reliable crash in our app; thanks!

Re: encoding/locale: note that up-thread there are env -i-based repro recipes that should have isolated the results from the contents of my environment, yet others fail to reproduce the crash. Curious if you know of other places besides the environment where encoding/locale/etc info might be sneaking into the mix (that might differ in your/my systems from those of others' who have failed to repro the crash).

lorentey · 2023-05-02T20:35:03Z

@fischman-bcny If the environment variable isn't set, Core Foundation has a legacy fallback to get its default encoding from the file ~/.CFUserTextEncoding, which usually does exist on macOS. Perhaps that's where the diverging configurations come from.

(Defaulting to anything other than UTF-8 doesn't make much sense to me these days, but fwiw it appears my Mac (in the US region) is still configured to prefer Mac Roman in this context. 🤷🏼‍♂️)

fischman-bcny · 2023-05-02T20:39:39Z

@lorentey nice! Indeed, I have a ~/.CFUserTextEncoding file containing 0x08000100:0x08000100. When I rename it, the crash from #63664 (comment) goes away, and when I put it back the crash is back.

lorentey · 2023-05-05T04:31:56Z

It turns out that the fault lies is outside of the Swift stdlib in this case, although it got triggered by Swift 5.7 in Ventura starting to trap on more out-of-bounds string accesses. We're tracking this issue separately (rdar://108765311); the fix will ship in a future OS release.

@fischman-bcny Do you happen to remember if you manually edited that file to set those contents, or if something configured it automatically?

fischman-bcny · 2023-05-05T17:09:36Z

@lorentey not sure, sorry.

AnthonyLatsis · 2023-05-16T03:57:05Z

It turns out that the fault lies is outside of the Swift stdlib in this case

@lorentey Is the issue with Foundation after all? If so, perhaps we should transfer the issue to the newly published Foundation repository?

lorentey · 2023-05-16T18:20:56Z

It is a Foundation issue, but it does not involve the code in swift-foundation. I think it's okay to keep the issue here for now.

brianmichel added bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. triage needed This issue needs more specific labels labels Feb 14, 2023

LucianoPAlmeida added standard library Area: Standard library umbrella String Area → standard library: The `String` type runtime The Swift Runtime run-time crash Bug → crash: Swift code crashed during execution and removed triage needed This issue needs more specific labels labels Feb 15, 2023

brianmichel closed this as completed Feb 15, 2023

AnthonyLatsis added crash Bug: A crash, i.e., an abnormal termination of software and removed runtime The Swift Runtime labels Feb 15, 2023

AnthonyLatsis added the Foundation label Feb 15, 2023

compnerd reopened this Feb 27, 2023

lorentey self-assigned this May 2, 2023

AnthonyLatsis added the swift 5.7 label May 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable To Append Two Emoji Strings On macOS Ventura's Swift #63664

Unable To Append Two Emoji Strings On macOS Ventura's Swift #63664

brianmichel commented Feb 14, 2023

WowbaggersLiquidLunch commented Feb 15, 2023

amonshiz commented Feb 15, 2023

danielpunkass commented Feb 15, 2023 •

edited

Loading

brianmichel commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023

fischman-bcny commented Feb 15, 2023 •

edited

Loading

danielpunkass commented Feb 15, 2023 •

edited

Loading

AnthonyLatsis commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023 •

edited

Loading

fischman-bcny commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023

fischman-bcny commented Mar 20, 2023

compnerd commented May 1, 2023

lorentey commented May 2, 2023

lorentey commented May 2, 2023 •

edited

Loading

lorentey commented May 2, 2023 •

edited

Loading

fischman-bcny commented May 2, 2023

lorentey commented May 2, 2023

fischman-bcny commented May 2, 2023

lorentey commented May 5, 2023

fischman-bcny commented May 5, 2023

AnthonyLatsis commented May 16, 2023

lorentey commented May 16, 2023

Unable To Append Two Emoji Strings On macOS Ventura's Swift #63664

Unable To Append Two Emoji Strings On macOS Ventura's Swift #63664

Comments

brianmichel commented Feb 14, 2023

WowbaggersLiquidLunch commented Feb 15, 2023

amonshiz commented Feb 15, 2023

danielpunkass commented Feb 15, 2023 • edited Loading

brianmichel commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023

fischman-bcny commented Feb 15, 2023 • edited Loading

danielpunkass commented Feb 15, 2023 • edited Loading

AnthonyLatsis commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023 • edited Loading

fischman-bcny commented Feb 15, 2023

AnthonyLatsis commented Feb 15, 2023

fischman-bcny commented Mar 20, 2023

compnerd commented May 1, 2023

lorentey commented May 2, 2023

lorentey commented May 2, 2023 • edited Loading

lorentey commented May 2, 2023 • edited Loading

fischman-bcny commented May 2, 2023

lorentey commented May 2, 2023

fischman-bcny commented May 2, 2023

lorentey commented May 5, 2023

fischman-bcny commented May 5, 2023

AnthonyLatsis commented May 16, 2023

lorentey commented May 16, 2023

danielpunkass commented Feb 15, 2023 •

edited

Loading

fischman-bcny commented Feb 15, 2023 •

edited

Loading

danielpunkass commented Feb 15, 2023 •

edited

Loading

AnthonyLatsis commented Feb 15, 2023 •

edited

Loading

lorentey commented May 2, 2023 •

edited

Loading

lorentey commented May 2, 2023 •

edited

Loading