Apple Keychain fails over ssh

[mattrabe]

Description

When I ssh into my Mac Mini (Big Sur) and run expo build:ios -t archive --release-channel dev I get an error when it tries to access the keychain: Security returned a non-successful error code: 36. Running the same command from that Mac directly (not ssh'd) works just fine.

Environment

I get identical output for this whether I am ssh'd in or working directly on the host machine.

⛵️ me@Hookena mobile-app <develop> $ expo diagnostics

  Expo CLI 4.0.17 environment info:
    System:
      OS: macOS 11.1
      Shell: 5.8 - /bin/zsh
    Binaries:
      Node: 15.5.0 - /usr/local/bin/node
      Yarn: 1.22.10 - ~/.nvm/versions/node/v15.5.0/bin/yarn
      npm: 7.3.0 - /usr/local/bin/npm
      Watchman: 4.9.0 - /usr/local/bin/watchman
    IDEs:
      Xcode: /undefined - /usr/bin/xcodebuild
    npmPackages:
      expo: ^39.0.0 => 39.0.3 
      react: 16.13.1 => 16.13.1 
      react-native: https://github.com/expo/react-native/archive/sdk-39.0.0.tar.gz => 0.63.2 
      react-navigation: ^4.4.2 => 4.4.2 
    npmGlobalPackages:
      expo-cli: 4.0.17
    Expo Workflow: managed

Host Machine:

2020 Mac Mini running M1 chip

macOS Big Sur v11.1

ssh setup was accomplished by turning on System Preferences -> Sharing -> Remote Login on the host, adding my client machine's key to authorized_keys, and then sshing from client to host using said key (NOT via password).

Expected Behavior

I expected that the keychain would be usable while I am running the build over ssh, or if that is not feasible, do not attempt to access the keychain and therefore do not err out.

Observed Behavior

Over ssh, I get the following error:

⛵️ me@Hookena mobile-app <develop> $ yarn build:dev
yarn run v1.22.10
$ yarn setEnv:dev expo build:ios -t archive --release-channel dev
$ ENV_NAME=development expo build:ios -t archive --release-channel dev

Checking if there is a build in progress...

Accessing credentials for fpcsteam in project brown-bear-car-wash-unlimited-wash-club
✔ Do you have access to the Apple account that will be used for submitting this app to the App Store? … yes
Please enter your Apple Developer Program account credentials. These credentials are needed to manage certificates, keys and provisioning profiles in your Apple Developer account.
The password is only used to authenticate with Apple and never stored on Expo servers
Learn more: https://bit.ly/2VtGWhU
✔ Apple ID: … me@hookena.com
✔ Password (for me@hookena.com): … ******
Saving Apple ID password to the local Keychain. Learn more https://docs.expo.io/distribution/security#keychain
Security returned a non-successful error code: 36
Error
    at new ErrorClass (/usr/local/lib/node_modules/expo-cli/node_modules/keychain/keychain.js:253:19)
    at ChildProcess.<anonymous> (/usr/local/lib/node_modules/expo-cli/node_modules/keychain/keychain.js:186:15)
    at ChildProcess.emit (node:events:376:20)
    at maybeClose (node:internal/child_process:1063:16)
    at Socket.<anonymous> (node:internal/child_process:449:11)
    at Socket.emit (node:events:376:20)
    at Pipe.<anonymous> (node:net:666:12)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Workaround

Still over ssh, if I set EXPO_NO_KEYCHAIN=1 and then run the same exact command as above, it works. However, this also has the undesirable consequence of removing the Apple ID from the keychain on the host machine.

Reproducible Demo

n/a

[brentvatne]

we use a pretty small wrapper around /usr/bin/security called keychain.js for this: https://github.com/drudge/node-keychain/blob/2daff235c867c9d9cb45845ceb560f6ea427828c/keychain.js

I expected that the keychain would be usable while I am running the build over ssh, or if that is not feasible, do not attempt to access the keychain and therefore do not err out.

i'm not sure we want to add any specific checks for this. it seems like it should be up to the user to decide if they want to opt out of keychain usage or use it, but i'm open to being convinced. that said, one thing we could do is catch an error and then ask users if they want to continue without keychain usage.

i'd suggest trying to make sure that you can run some security commands from your user, and investigating what error code 36 for security(1) means. as far as i can tell this is the expected behavior from expo-cli and the issue is with the user/os config

[mattrabe]

Thank you! I will look into those tomorrow.

[mattrabe]

I have been playing with the security cli command, but it is producing unexpected results - and unfortunately not in a way that seems to point to the cause of my problem here.

When sitting at the host machine (not ssh'd), I cannot access the password for expo via the security cli:

⛵️ mattrabe@Hookena ~ $ security find-internet-password -s expo -g
security: SecKeychainSearchCopyNext: The specified item could not be found in the keychain.
⛵️ mattrabe@Hookena ~ $ security find-generic-password -s expo -g
security: SecKeychainSearchCopyNext: The specified item could not be found in the keychain.
⛵️ mattrabe@Hookena ~ $ security find-internet-password -s http://expo.io -g
security: SecKeychainSearchCopyNext: The specified item could not be found in the keychain.
⛵️ mattrabe@Hookena ~ $ security find-generic-password -s http://expo.io -g
security: SecKeychainSearchCopyNext: The specified item could not be found in the keychain.

However, running the expo build:ios command while also sitting at the host machine (not ssh'd) DOES work, and DOES successfully access the password from the keychain. And those items ARE present in my Keychain:

If I even just run a keychain dump for those items, they are not present:

⛵️ mattrabe@Hookena mobile-app <develop> $ security dump-keychain | grep expo
⛵️ mattrabe@Hookena mobile-app <develop> $

Greping for other values on that same dump does produce results.

So what am I missing here?

I did find this StackExchange, which is similar in that it refers to the same 36 error status happening when over ssh. However, it's quite old, the interface has changed, and I was unable to find a resolution by following the answers. I did try security unlock-keychain (and numerous variations thereof), and they did not lead to a resolution. It did clarify for me that there are multiple Keychains, though - and I see that the expo passwords are stored in the iCloud keychain (screenshot above). I wonder if that is problematic. There is no "icloud" keychain file in my user's dir:

⛵️ mattrabe@Hookena mobile-app <develop> $ ls -l ~/Library/Keychains
total 352
drwx------  11 mattrabe  staff     352 Jan  8 08:37 80709CDB-B490-568B-B3E4-573BE3E3F601
-rw-r--r--@  1 mattrabe  staff  154344 Jan  8 08:51 login.keychain-db
-rw-------   1 mattrabe  staff   23804 Jan  6 11:42 metadata.keychain-db

Interestingly, the expo items are stored in the "Local Items" keychain on a different machine I use. Both machines are able to successfully build at cli using expo build:ios - it's only when I ssh from A to B and try building on B that I get the error.

A few questions that I don't have answers to:

1. What is the name of the service in the Keychain that expo needs to retrieve the password from? Is "expo" not in the name? I see several "expo.io" items in my Keychain GUI, but nothing tangible comes up for "Apple Developer" or even just "Developer".
2. Does expo use find-internet-password or find-generic-password? The expo items are listed in my Keychain GUI as "Web form password", which doesn't appear to be either of those things.
3. Do I need to specify the keychain the item is stored in when making a security request? If so, how? I don't see any information on this in security find-internet-password --help or security find-generic-password --help, so I'm guessing no.
4. Is having this stored in iCloud keychain problematic?

[brentvatne]

@mattrabe - i don't really know that much about this personally and don't have much time to investigate the edge case that you're encountering, but hopefully by reading the source you can answer your questions: https://github.com/expo/expo-cli/search?q=keychain

keychain service name is here:

expo-cli/packages/expo-cli/src/appleApi/authenticate.ts

Lines 250 to 258 in d5a20f0

	/**
	* Returns the same prefix used by Fastlane in order to potentially share access between services.
	* [Cite. Fastlane](https://github.com/fastlane/fastlane/blob/f831062fa6f4b216b8ee38949adfe28fc11a0a8e/credentials_manager/lib/credentials_manager/account_manager.rb#L8).
	*
	* @param appleId email address
	*/
	function getKeychainServiceName(appleId: string): string {
	return `deliver.${appleId}`;
	}

[mattrabe]

I’ll keep digging, but maybe @EvanBacon could shed some light?

[brentvatne]

i think these are questions related to keychain and not specific to expo-cli, so i don't think this is necessarily the best place to ask.

What is the name of the service in the Keychain that expo needs to retrieve the password from? Is "expo" not in the name? I see several "expo.io" items in my Keychain GUI, but nothing tangible comes up for "Apple Developer" or even just "Developer".

i think this is what we can answer for you - it should be under deliver.<your-apple-id-email> as per the code i shared above. the expo.io items are from the website

as for keychain type:

expo-cli/packages/expo-cli/src/appleApi/keychain.ts

Line 5 in cc4613d

const KEYCHAIN_TYPE = 'internet';

[mattrabe]

Yea, I'm finding answers to those questions via the source - thanks for pointing me toward that!

While the issue is certainly related to keychain, I'm thinking there is room for improvement here on expo-cli, in that the handling of this case is not great. expo-cli just fails outright in this scenario, where it could at least allow the user to manually enter the password to complete the build. I'll grant that this is probably an edge case.

Now that I know what items in keychain I am looking for, I am able to see that I am in fact getting different responses when ssh'd vs not ssh'd. I'll continue tracking that down and intend to open a PR if I can find a good method to work around this.

[brentvatne]

i agree we can handle it better! catching the error and letting users proceed w/o keychain seems like the most simple catchall approach

[mattrabe]

Update:

I see that I am getting different responses from the security program used under the hood to access the keychain items when I am ssh'd vs when I am not ssh'd. This is perhaps not surprising, as apple may restrict access to the keychain when ssh'd.

When I am NOT ssh'd, I CAN retrieve the password:

⛵️ mattrabe@Kanaloa ~ $ security find-internet-password -s deliver.<my apple id email> -a <my apple id email> -w
<right here it DOES output the password>
⛵️ mattrabe@Kanaloa ~ $

But when I am ssh'd into the same machine, using the same macOS user account, I get NO OUTPUT for the same command:

⛵️ mattrabe@Kanaloa ~ $ security find-internet-password -s deliver.<my apple id email> -a <my apple id email> -w
⛵️ mattrabe@Kanaloa ~ $

(Note that node-keychain appears to use -g instead of -w, but the behavior is the same for -g so I am just using -w here for brevity of output)

And here are the answers to my questions from a few comments ago:

1. What is the name of the service in the Keychain that expo needs to retrieve the password from?
   • deliver.<your Apple ID email>
2. Does expo use find-internet-password or find-generic-password?
   • find-internet-password
3. Do I need to specify the keychain the item is stored in when making a security request?
   • No.
4. Is having this stored in iCloud keychain problematic?
   • n/a. I was looking at the wrong keychain items, the correct items (deliver.<your apple id email) are stored in the "local" keychain.

---

So that confirms that behavior is different when ssh'd. I'll see if I can put together a good way to catch that, and allow the user to continue on with the build.

[gCardinal]

While the CLI not breaking when it can't write to the keychain is useful, for people that would like the keychain to be writable while connected via SSH, you can run security unlock-keychain -p "enter password" in your script. After, any command that tries to write or access they keychain should be successful.

Very useful for CIs and such.

[mattrabe]

@gCardinal Thanks for the tip! I just tested it and it did work in my scenario. Agreed that this would be great for CIs. Using security unlock-keychain (without -p "<password>") also works, and prompts for a password in real-time.

Still glad to see that this PR was merged, since it doesn't require documentation/remembering things.

[github-actions]

This issue is stale because it has been open for 60 days with no activity. If there is no activity in the next 7 days, the issue will be closed.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale. Please open a new issue if you believe you are encountering a related problem.