rstudio 1.2 has runtime dependency on homebrew dynlib

[jeroen]

System details

RStudio Edition : Desktop
RStudio Version :  1.2.1268
OS Version      :  MacOS
R Version       : latest

Summary

The rstudio callback executable used for git and ssh password entry seems to have been linked to openssl dynamic libraries from homebrew. Therefore the password prompt will not work if the user does not have homebrew installed:

otool -L /Applications/RStudio.app/Contents/MacOS/rpostback

Gives

/Applications/RStudio.app/Contents/MacOS/rpostback:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 775.19.0)
	/usr/local/opt/openssl/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.5.0)

Note the stuff under /usr/local/opt is homebrew specific and won't be installed on most machines. I think the intention is to link to openssl in/Applications/RStudio.app/Contents/Frameworks ? You can use install_name_tool to change the path:

install_name_tool -change \
  /usr/local/opt/openssl/lib/libssl.1.0.0.dylib \
  /Applications/RStudio.app/Contents/Frameworks/libssl.1.0.0.dylib \
  /Applications/RStudio.app/Contents/MacOS/rpostback

install_name_tool -change \
  /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib \
  /Applications/RStudio.app/Contents/Frameworks/libcrypto.1.0.0.dylib \
  /Applications/RStudio.app/Contents/MacOS/rpostback
  

Steps to reproduce the problem

If you have homebrew, first makes sure openssl is not installed:

brew remove openssl --ignore-dependencies

And then for example:

processx::run("/Applications/RStudio.app/Contents/MacOS/postback/rpostback-askpass", "test")

[jmcphers]

We bundle OpenSSL on MacOS, but I think the actual link path is to Homebrew's OpenSSL (which we compile against at build time). We probably just need to fix up rpostback-askpass in the same way that we fix up diagnostics here.

rstudio/src/cpp/diagnostics/CMakeLists.txt

Lines 68 to 76 in 8af7304

	# if doing a package build on MacOS, reroute the OpenSSL libraries to our bundled copies
	if(APPLE AND RSTUDIO_PACKAGE_BUILD)
	find_package(OpenSSL REQUIRED QUIET)
	foreach(lib ${OPENSSL_CRYPTO_LIBRARY} ${OPENSSL_SSL_LIBRARY})
	get_filename_component(LIB_DIR ${lib} PATH)
	execute_process(COMMAND readlink ${lib} OUTPUT_VARIABLE LIB_FILE OUTPUT_STRIP_TRAILING_WHITESPACE)
	add_custom_command (TARGET diagnostics POST_BUILD
	COMMAND install_name_tool -change ${LIB_DIR}/${LIB_FILE} @executable_path/../Frameworks/${LIB_FILE} ${CMAKE_CURRENT_BINARY_DIR}/diagnostics)
	endforeach()