Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) #68785

rdj · 2023-09-27T00:50:34Z

Description
Swift script that imports AppKit fails with "JIT session error: Symbols not found".

By "swift script" I mean a swift source file that is run directly with /usr/bin/swift, xcrun swift, or as an executable script with a shebang line like #!/usr/bin/env swift.

Using swiftc on the same source file compiles cleanly and the executable works as expected.

Steps to reproduce

 $ printf 'import AppKit\n\nprint( NSScreen.main!.frame.size )' > a.swift && swift a.swift
JIT session error: Symbols not found: [ _OBJC_CLASS_$_NSScreen ]
Failed to materialize symbols: { (main, { _$sSoMXM, __swift_FORCE_LOAD_$_swiftDarwin_$_a, __swift_FORCE_LOAD_$_swiftObjectiveC_$_a, __swift_FORCE_LOAD_$_swiftDispatch_$_a, __swift_FORCE_LOAD_$_swiftXPC_$_a, _$ss5print_9separator10terminatoryypd_S2StFfA0_, _$ss27_finalizeUninitializedArrayySayxGABnlF, _got.$s12CoreGraphics7CGFloatVMn, __swift_FORCE_LOAD_$_swiftos_$_a, _symbolic _____ So6CGSizeV, _symbolic _____ 12CoreGraphics7CGFloatV, _$ss5print_9separator10terminatoryypd_S2StFfA1_, _$sSo6CGSizeVwst, __swift_FORCE_LOAD_$_swiftCoreImage_$_a, _$sSa12_endMutationyyF, __swift_FORCE_LOAD_$_swiftCoreGraphics_$_a, _$sSo6CGSizeVMF, $.a.__inits.0, __swift_FORCE_LOAD_$_swiftQuartzCore_$_a, __swift_FORCE_LOAD_$_swiftUniformTypeIdentifiers_$_a, __swift_FORCE_LOAD_$_swiftMetal_$_a, _$sSo6CGSizeVMn, _$sSo6CGSizeVMf, __swift_FORCE_LOAD_$_swiftCoreFoundation_$_a, _$sSo6CGSizeVML, _$sSo6CGSizeVMB, __swift_FORCE_LOAD_$_swiftFoundation_$_a, _main, _$sSo6CGSizeVwet, _$sSo6CGSizeVMa, __swift_FORCE_LOAD_$_swiftOSLog_$_a, _$sSo6CGSizeVWV, ___swift_noop_void_return, ___swift_memcpy16_8, __swift_FORCE_LOAD_$_swiftIOKit_$_a }) }

 $ swiftc a.swift && ./a
(2560.0, 1440.0)

Expected behavior
Should work when run directly with swift or as an executable with #!/usr/bin/env swift.

Environment
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx14.0

Xcode 15.0
Build version 15A240d

Related Issues
Similar to #59354

The text was updated successfully, but these errors were encountered:

ausdrew · 2023-09-27T11:01:33Z

+1 - Sonoma has bricked my lil' command line scripts I was using fine on Ventura

Except with import Cocoa.

swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1)
Target: arm64-apple-macosx14.0

JIT session error: Symbols not found: [ _OBJC_CLASS_$_NSPasteboard, _NSPasteboardTypeString ]

rdj · 2023-09-27T21:23:55Z

Also works as expected from REPL:

$ swift repl
Welcome to Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1).
Type :help for assistance.
  1> import AppKit
  2> let screenSize = NSScreen.main!.frame.size
screenSize: CGSize = (width = 2560, height = 1440)

zxdposter · 2023-11-14T06:54:52Z

@ausdrew xcrun -sdk macosx swiftc YourScript.swift -o YourScript. from, its work for me.

tib · 2023-11-14T15:10:32Z

@ausdrew xcrun -sdk macosx swiftc YourScript.swift -o YourScript. from, its work for me.

That command is invoking the Swift compiler and instead of running the script directly using the swift command. @rdj also mentioned that it's possible to compile the source file...

Based on my experience, this issue still exists. Hopefully someone from Apple will see this and provide a fix real soon. 🤞 🙏

skonesam · 2023-11-17T21:49:14Z

I have many similar little scripts that fail from importing Cocoa.

Started off as an easy way to get familiar with the language. Now that they're failing, I realize I relied on them a LOT!

hvge · 2023-11-28T08:48:22Z

I have the same problem when script is using CryptoKit framework (except the different missing symbols, of course).

jessesquires · 2023-12-06T21:37:57Z

I am also experiencing this issue.

Xcode 15.1 (22502)
macOS Sonoma 14.1.2 (23B2091)

dempseyatgithub · 2023-12-07T17:56:08Z

I am also seeing this issue when trying to import Core Data in a script that has worked for years.

Since this may be a macOS / Swift integration issue and not a pure Swift issue, I have filed a Feedback with Apple:
'FB13453942 Regression: Scripts written in Swift fail when importing system frameworks'

dempseyatgithub · 2023-12-07T18:08:38Z

@rdj Would you be able to add the 'regression' tag to this issue? Hopefully that would help raise its visibility.

rdj · 2023-12-07T18:32:57Z

@dempseyatgithub Nope, I think tagging issues is a privileged op. I'm just a random internet person with no special powers.

benlangmuir · 2023-12-08T22:22:43Z

This appears to be indirectly related to libswiftAppKit code being merged back into AppKit. Before that SDK change, we would autolink libswiftAppKit. That caused us to do:

dlopen("libswiftAppKit.dylib", ...)

that succeeds, and it pulls in AppKit.framework as a dependency. On 14.0 and later, we autolink AppKit.framework not the overlay dylib, and call

dlopen("AppKit.framework/AppKit", ...)

which fails. I think the right fix would be for us to actually use the path /System/Library/Frameworks/AppKit.framewok/AppKit.

As an aside, after the SDK change we needed the following e759007 so that running on macOS < 14.0 but building against a 14+ SDK we get the right behaviour. But that's not directly relevant here.

apple/swift#68785

Savjee · 2024-01-19T13:12:37Z

I found a workaround. If you manually set the system framework search path, everything works:

swift -Fsystem /System/Library/Frameworks path_to_script.swift

jessesquires · 2024-01-19T19:08:18Z

@Savjee Hm, that does not seem to work for all use cases. I have a script that imports Foundation and AppKit

Invoking it using -Fsystem /System/Library/Frameworks produces the following errors:

<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "Headers/AppKit.h"
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/System/Library/Frameworks/AppKit.framework/Headers/AppKit.h:10:9: error: 'Foundation/Foundation.h' file not found
#import <Foundation/Foundation.h>
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk/System/Library/Frameworks/AppKit.framework/Headers/AppKit.h:10:9: note: did not find header 'Foundation.h' in framework 'Foundation' (loaded from '/System/Library/Frameworks')
#import <Foundation/Foundation.h>
        ^
<unknown>:0: error: could not build Objective-C module 'AppKit'

…times. Add DYLD_FRAMEWORK_PATH=/System/Library/Frameworks to the environment when constructing swift-frontend invocations for the interpreter so that dlopen can find autolinked frameworks. This is intended to fix #68785. As a follow-up: don't expand runtime paths to find the path to the runtime in libImmediate, but instead hard-code "/usr/lib/swift/libswiftCore.dylib" and allow any overrides to be found via DYLD_LIBRARY_PATH (this ensures that dyld treats whichever libswiftCore.dylib is found as override for the one in the shared cache, which doesn't happen if dlopen is passed a non-standard path).

rdj · 2024-01-22T17:55:50Z

@lhames So should just setting the environment have the same effect, cause it does not resolve the repro from the original bug:

$ printf 'import AppKit\n\nprint( NSScreen.main!.frame.size )' > a.swift
$ DYLD_FRAMEWORK_PATH=/System/Library/Frameworks swift a.swift
JIT session error: Symbols not found: [ _OBJC_CLASS_$_NSScreen ]
Failed to materialize symbols: { (main, { _got.$s12CoreGraphics7CGFloatVMn, __swift_FORCE_LOAD_$_swiftCoreGraphics_$_a, _symbolic _____ 12CoreGraphics7CGFloatV, ___swift_noop_void_return, __swift_FORCE_LOAD_$_swiftMetal_$_a, _$ss5print_9separator10terminatoryypd_S2StFfA0_, __swift_FORCE_LOAD_$_swiftQuartzCore_$_a, ___swift_memcpy16_8, _$ss27_finalizeUninitializedArrayySayxGABnlF, __swift_FORCE_LOAD_$_swiftos_$_a, _$ss5print_9separator10terminatoryypd_S2StFfA1_, __swift_FORCE_LOAD_$_swiftDarwin_$_a, __swift_FORCE_LOAD_$_swiftObjectiveC_$_a, __swift_FORCE_LOAD_$_swiftDispatch_$_a, __swift_FORCE_LOAD_$_swiftXPC_$_a, _$sSo6CGSizeVMB, _$sSo6CGSizeVML, __swift_FORCE_LOAD_$_swiftCoreFoundation_$_a, $.a.__inits.0, _$sSo6CGSizeVMF, __swift_FORCE_LOAD_$_swiftFoundation_$_a, _$sSoMXM, _$sSo6CGSizeVWV, __swift_FORCE_LOAD_$_swiftOSLog_$_a, _symbolic _____ So6CGSizeV, _$sSo6CGSizeVMf, _$sSo6CGSizeVMn, __swift_FORCE_LOAD_$_swiftIOKit_$_a, _$sSo6CGSizeVwet, __swift_FORCE_LOAD_$_swiftUniformTypeIdentifiers_$_a, _$sSa12_endMutationyyF, __swift_FORCE_LOAD_$_swiftCoreImage_$_a, _$sSo6CGSizeVMa, _$sSo6CGSizeVwst, _main }) }

lhames · 2024-01-22T18:23:49Z

@lhames So should just setting the environment have the same effect, cause it does not resolve the repro from the original bug: ...

@rdj I've been testing on top-of-tree. It's possible that there's something else going on on 5.9.

Could you try running DYLD_PRINT_SEARCHING=1 DYLD_FRAMEWORK_PATH=/System/Library/Frameworks swift a.swift and attach the output?

rdj · 2024-01-22T18:55:14Z

Hmm, no effect.

Judging from man dyld, probably because "Note: If System Integrity Protection is enabled, these environment variables are ignored when executing binaries protected by System Integrity Protection." Which means it is also ignoring the DYLD_FRAMEWORK_PATH var, so maybe the fix will work.

Ok, I can get it to listen to the DYLD vars by invoking the xcode-installed swift directly. At which point this does seem to fix the problem:

$ DYLD_FRAMEWORK_PATH=/System/Library/Frameworks /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift a.swift
(2560.0, 1440.0)

Oddly, it does not work through xcrun, but using the -find path does. I guess xcrun is SIP protected and it gets stripped out of the environment before calling into swift.

$ DYLD_FRAMEWORK_PATH=/System/Library/Frameworks xcrun swift a.swift 
JIT session error: Symbols not found: [ _OBJC_CLASS_$_NSScreen ]
Failed to materialize symbols: { (main, { _$ss5print_9separator10terminatoryypd_S2StFfA0_, _symbolic _____ 12CoreGraphics7CGFloatV, _$ss5print_9separator10terminatoryypd_S2StFfA1_, ___swift_memcpy16_8, __swift_FORCE_LOAD_$_swiftQuartzCore_$_a, _$ss27_finalizeUninitializedArrayySayxGABnlF, _got.$s12CoreGraphics7CGFloatVMn, ___swift_noop_void_return, _$sSo6CGSizeVMB, __swift_FORCE_LOAD_$_swiftDarwin_$_a, _$sSo6CGSizeVMF, $.a.__inits.0, __swift_FORCE_LOAD_$_swiftXPC_$_a, __swift_FORCE_LOAD_$_swiftos_$_a, __swift_FORCE_LOAD_$_swiftObjectiveC_$_a, __swift_FORCE_LOAD_$_swiftCoreFoundation_$_a, _symbolic _____ So6CGSizeV, __swift_FORCE_LOAD_$_swiftIOKit_$_a, _$sSo6CGSizeVwst, __swift_FORCE_LOAD_$_swiftDispatch_$_a, _$sSa12_endMutationyyF, __swift_FORCE_LOAD_$_swiftFoundation_$_a, _$sSo6CGSizeVML, __swift_FORCE_LOAD_$_swiftCoreImage_$_a, _$sSo6CGSizeVMf, _$sSo6CGSizeVwet, _$sSo6CGSizeVMn, __swift_FORCE_LOAD_$_swiftMetal_$_a, _$sSoMXM, __swift_FORCE_LOAD_$_swiftUniformTypeIdentifiers_$_a, _$sSo6CGSizeVWV, __swift_FORCE_LOAD_$_swiftOSLog_$_a, _$sSo6CGSizeVMa, __swift_FORCE_LOAD_$_swiftCoreGraphics_$_a, _main }) }

$ DYLD_FRAMEWORK_PATH=/System/Library/Frameworks $(xcrun -find swift) a.swift 
(2560.0, 1440.0)

Thank, @lhames, it does seem like this will fix the top-post repro on this issue.

rdj · 2024-01-22T19:02:44Z

So for others on the thread, a workaround — if you have xcode installed — is to change your shebang line:

#!/usr/bin/env DYLD_FRAMEWORK_PATH=/System/Library/Frameworks /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift

lhames · 2024-01-22T19:09:30Z

@rdj Ahh. I need to do some more testing on a SIP enabled machine.

Thank, @lhames, it does seem like this will fix the top-post repro on this issue.

I think this is a reasonable interim solution, but I'm not sure it's the best solution. I'm happy to keep discussing.

jessesquires · 2024-01-22T23:20:19Z

So for others on the thread, a workaround — if you have xcode installed — is to change your shebang line:

@rdj thank you for this! 🙌🏼 Confirmed it works for me.

This line was accidentally dropped from 06384be, which also contained a related fix to libImmediate (see that commit for details). Resolves: apple#68785

…times. Add DYLD_FRAMEWORK_PATH=/System/Library/Frameworks to the environment when constructing swift-frontend invocations for the interpreter so that dlopen can find autolinked frameworks. This is intended to fix apple#68785. As a follow-up: don't expand runtime paths to find the path to the runtime in libImmediate, but instead hard-code "/usr/lib/swift/libswiftCore.dylib" and allow any overrides to be found via DYLD_LIBRARY_PATH (this ensures that dyld treats whichever libswiftCore.dylib is found as override for the one in the shared cache, which doesn't happen if dlopen is passed a non-standard path).

This line was accidentally dropped from 06384be, which also contained a related fix to libImmediate (see that commit for details). Resolves: apple#68785

rdj added bug A deviation from expected or documented behavior. Also: expected but undesirable behavior. triage needed This issue needs more specific labels labels Sep 27, 2023

rdj changed the title ~~Swift script importing AppKit fails with Symbols not found~~ Swift script importing AppKit fails with Symbols not found (macOS 14) Sep 27, 2023

ausdrew mentioned this issue Sep 27, 2023

Doesn't work on Mac OS Sonoma ausdrew/ShakeWellerer#1

Closed

jacoblange-dev mentioned this issue Nov 22, 2023

Failing to start Raycast command scripts - MacOS Sonoma raycast/script-commands#887

Closed

rdj changed the title ~~Swift script importing AppKit fails with Symbols not found (macOS 14)~~ Swift script importing Cocoa framworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) Nov 28, 2023

rdj changed the title ~~Swift script importing Cocoa framworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14)~~ Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) Nov 28, 2023

AnthonyLatsis added regression swift 5.9 labels Dec 7, 2023

jessesquires added a commit to jessesquires/jessesquires.com that referenced this issue Dec 15, 2023

fix script (workaround)

f391117

apple/swift#68785

lhames mentioned this issue Jan 19, 2024

For scripts, use DYLD_FRAMEWORK/LIBRARY_PATH to find frameworks / run… #71004

Merged

lhames closed this as completed in #71004 Jan 22, 2024

lhames reopened this Jan 22, 2024

lhames mentioned this issue Jan 24, 2024

Add missing "/System/Library/Frameworks" path to DYLD_FRAMEWORK_PATH. #71119

Merged

lhames closed this as completed in #71119 Jan 25, 2024

lhames mentioned this issue Jan 25, 2024

For scripts, use DYLD_FRAMEWORK/LIBRARY_PATH to find frameworks / runtimes. #71158

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) #68785

Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) #68785

rdj commented Sep 27, 2023

ausdrew commented Sep 27, 2023

rdj commented Sep 27, 2023

zxdposter commented Nov 14, 2023 •

edited

tib commented Nov 14, 2023

skonesam commented Nov 17, 2023 •

edited

hvge commented Nov 28, 2023 •

edited

jessesquires commented Dec 6, 2023

dempseyatgithub commented Dec 7, 2023

dempseyatgithub commented Dec 7, 2023

rdj commented Dec 7, 2023

benlangmuir commented Dec 8, 2023

Savjee commented Jan 19, 2024

jessesquires commented Jan 19, 2024

rdj commented Jan 22, 2024

lhames commented Jan 22, 2024 •

edited

rdj commented Jan 22, 2024

rdj commented Jan 22, 2024 •

edited

lhames commented Jan 22, 2024

jessesquires commented Jan 22, 2024

Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) #68785

Swift script importing Cocoa frameworks (Cocoa, AppKit, CryptoKit) fails with Symbols not found (macOS 14) #68785

Comments

rdj commented Sep 27, 2023

ausdrew commented Sep 27, 2023

rdj commented Sep 27, 2023

zxdposter commented Nov 14, 2023 • edited

tib commented Nov 14, 2023

skonesam commented Nov 17, 2023 • edited

hvge commented Nov 28, 2023 • edited

jessesquires commented Dec 6, 2023

dempseyatgithub commented Dec 7, 2023

dempseyatgithub commented Dec 7, 2023

rdj commented Dec 7, 2023

benlangmuir commented Dec 8, 2023

Savjee commented Jan 19, 2024

jessesquires commented Jan 19, 2024

rdj commented Jan 22, 2024

lhames commented Jan 22, 2024 • edited

rdj commented Jan 22, 2024

rdj commented Jan 22, 2024 • edited

lhames commented Jan 22, 2024

jessesquires commented Jan 22, 2024

zxdposter commented Nov 14, 2023 •

edited

skonesam commented Nov 17, 2023 •

edited

hvge commented Nov 28, 2023 •

edited

lhames commented Jan 22, 2024 •

edited

rdj commented Jan 22, 2024 •

edited