Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apple Silicon ARM64 : Invalid access: Can not convert empty value. #3331

Closed
asmaan opened this issue Jun 28, 2021 · 18 comments · Fixed by #4394
Closed

Apple Silicon ARM64 : Invalid access: Can not convert empty value. #3331

asmaan opened this issue Jun 28, 2021 · 18 comments · Fixed by #4394

Comments

@asmaan
Copy link

asmaan commented Jun 28, 2021

I have compiled poco for arm64 for Apple Silicon Native support. But for arm devices JSON parsing fails even for simplest json contracts. Please see below code snippet.

std::string json = "{\"clientConfig\" : \"Franky\"}";
        Poco::JSON::Parser jsonParser;
        Poco::Dynamic::Var jsonObject = jsonParser.parse(json);
        Poco::JSON::Object::Ptr jsonPtr = jsonObject.extract<Poco::JSON::Object::Ptr>();
        std::string testStr = jsonPtr->getValue<std::string>("clientConfig"); -> Here I get that exception.

If I check jsonPtr->get("clientConfig").isEmpty() , it returns true as well.

I tried to compile with -fvisibility=hidden as well but same result.
If I compiled for x86_64 and run on Silicon Macs(through rosetta), it work fine. On iOS it works fine too.
If I run in debug mode on Apple Silicon Mac, it works fine, issue is only in release mode.

Any ideas what's causing this?

@asmaan
Copy link
Author

asmaan commented Jun 28, 2021

@obiltschnig Can you please take a quick look?

@asmaan asmaan changed the title ARM64 : Invalid access: Can not convert empty value. Apple Silicon ARM64 : Invalid access: Can not convert empty value. Jun 28, 2021
@obiltschnig obiltschnig self-assigned this Jun 28, 2021
@obiltschnig obiltschnig added this to the Release 1.11.1 milestone Jun 28, 2021
@obiltschnig
Copy link
Member

I don't have an Apple Silicon Mac available. I can do a quick test on iOS maybe. I remember a similar issue on Android.

@obiltschnig
Copy link
Member

Probably related: #2844

@asmaan
Copy link
Author

asmaan commented Jun 28, 2021

Can you please guide me to android issue? I found few but they were not specific to this particular error.
On iOS it works for me, issue is specific to ARM based Macs only.
Another thing to add, If I run in debug mode, it works fine, issue is only in release mode.(Will put in description as well)

@asmaan
Copy link
Author

asmaan commented Jun 28, 2021

@obiltschnig
I looked at the android issue and it looks similar to that.But in my case exception is different. Any workarounds you can suggest?

@asmaan
Copy link
Author

asmaan commented Jun 28, 2021

My suspicion is that this issue is under this snippet in Var.h where pHolder is nil

template <typename T>
	T convert() const
		/// Invoke this method to perform a safe conversion.
		///
		/// Example usage:
		///     Var any("42");
		///     int i = any.convert<int>();
		///
		/// Throws a RangeException if the value does not fit
		/// into the result variable.
		/// Throws a NotImplementedException if conversion is
		/// not available for the given type.
		/// Throws InvalidAccessException if Var is empty.
	{
		VarHolder* pHolder = content();

		if (!pHolder)
			throw InvalidAccessException("Can not convert empty value.");

		if (typeid(T) == pHolder->type()) return extract<T>();

		T result;
		pHolder->convert(result);
		return result;
	}

@asmaan
Copy link
Author

asmaan commented Jun 28, 2021

Finally managed to figure it out. Set ENABLE_TESTABILITY to NO in your target's Release mode build settings . This will fix the issue. I don't know why poco needs this privilege to run. @obiltschnig Can you please help me understand why poco need this entitlement?

@obiltschnig
Copy link
Member

Could you provide the detailed clang command line XCode uses in release build with both ENABLE_TESTABILITY set to on and off. It likely has to do with some compiler setting (-fvisibility maybe) that influence the way RTTI works across dylib boundaries.

@asmaan
Copy link
Author

asmaan commented Jun 29, 2021

I can only see export ENABLE_TESTABILITY\=YES , no trace of this flag in clang command line. See below clan command.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -x objective-c++ -target arm64-apple-macos10.14 -fmessage-length=0 -fdiagnostics-show-note-include-stack -fmacro-backtrace-limit=0 -std=gnu++1z -stdlib=libc++ -fobjc-arc -fmodules -gmodules -fmodules-cache-path=/Users/testuser/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -fmodules-prune-interval=86400 -fmodules-prune-after=345600 -fbuild-session-file=/Users/testuser/Library/Developer/Xcode/DerivedData/ModuleCache.noindex/Session.modulevalidation -fmodules-validate-once-per-build-session -Wnon-modular-include-in-framework-module -Werror=non-modular-include-in-framework-module -Wno-trigraphs -fpascal-strings -Os -fno-common -Wno-missing-field-initializers -Wno-missing-prototypes -Werror=return-type -Wunreachable-code -Wno-implicit-atomic-properties -Werror=deprecated-objc-isa-usage -Wno-objc-interface-ivars -Werror=objc-root-class -Wno-arc-repeated-use-of-weak -Wimplicit-retain-self -Wno-non-virtual-dtor -Wno-overloaded-virtual -Wno-exit-time-destructors -Wduplicate-method-match -Wno-missing-braces -Wparentheses -Wswitch -Wunused-function -Wno-unused-label -Wno-unused-parameter -Wunused-variable -Wunused-value -Wempty-body -Wuninitialized -Wconditional-uninitialized -Wno-unknown-pragmas -Wno-shadow -Wno-four-char-constants -Wno-conversion -Wconstant-conversion -Wint-conversion -Wbool-conversion -Wenum-conversion -Wno-float-conversion -Wnon-literal-null-conversion -Wobjc-literal-conversion -Wshorten-64-to-32 -Wno-newline-eof -Wno-selector -Wno-strict-selector-match -Wundeclared-selector -Wdeprecated-implementations -Wno-c++11-extensions -DDEVELOPMENT_TEAM="TTTTTTTT" -Dtray=1 -DNS_BLOCK_ASSERTIONS=1 -DOBJC_OLD_DISPATCH_PROTOTYPES=0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fstrict-aliasing -Wprotocol -Wdeprecated-declarations -Winvalid-offsetof -g -Wno-sign-conversion -Winfinite-recursion -Wmove -Wcomma -Wblock-capture-autoreleasing -Wno-strict-prototypes -Wrange-loop-analysis -Wno-semicolon-before-method-body -iquote /

BuildProductsPath/Release -D_DARWIN_UNLIMITED_SELECT -DFD_SETSIZE=65536 -MMD -MT dependencies -MF

@obiltschnig
Copy link
Member

Still no access to Apple Silicon Mac to reproduce. Moving to 1.13.

@matejk
Copy link
Contributor

matejk commented Mar 22, 2023

Latest version from devel works properly on Apple Silicon (M1 Pro Max) for the snippet provided by the submitter.

% c++ --version
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.3.0
Thread model: posix

% sw_vers 
ProductName:		macOS
ProductVersion:		13.2.1
BuildVersion:		22D68

@matejk
Copy link
Contributor

matejk commented Mar 22, 2023

However few other tests fail:

There were 3 errors: 
 1: N7CppUnit10TestCallerI8JSONTestEE.testValidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 2: N7CppUnit10TestCallerI8JSONTestEE.testInvalidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 3: N7CppUnit10TestCallerI8JSONTestEE.testInvalidUnicodeJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1

@bas524
Copy link
Contributor

bas524 commented Apr 17, 2023

However few other tests fail:

There were 3 errors: 
 1: N7CppUnit10TestCallerI8JSONTestEE.testValidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 2: N7CppUnit10TestCallerI8JSONTestEE.testInvalidJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1
 3: N7CppUnit10TestCallerI8JSONTestEE.testInvalidUnicodeJanssonFiles
    "N4Poco17NotFoundExceptionE:
Not found: POCO_BASE"
    in "<unknown>", line -1

It is not an error, please set environment variable POCO_BASE, which contains path to the poco source directory

@xzzh999
Copy link

xzzh999 commented Sep 28, 2023

in macos mini M1 arm64, i found the json function Array::Ptr Object::getArray always get empty array , and this test failed:
it->second.type() == typeid(Array::Ptr)
but i print the it->second.type().name() is eq typeid(Array::Ptr).name(),
even add -frtti flag still not working.
strange ...

i just fix this use:

Array::Ptr Object::getArray(const std::string& key) const
{
	ValueMap::const_iterator it = _values.find(key);

	// FIXME:
	// if ((it != _values.end()) && (it->second.type() == typeid(Array::Ptr)))
	if ((it != _values.end()) && (std::string(it->second.type().name()) == typeid(Array::Ptr).name()))
	{
		return it->second.extract<Array::Ptr>();
	}

	return 0;
}

@aleks-f aleks-f self-assigned this Oct 26, 2023
@matejk
Copy link
Contributor

matejk commented Nov 13, 2023

I added the exact code as provided in the description to the unit test.

It works properly on M1 MacBook:

% c++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.1.0
Thread model: posix

% sw_vers
ProductName:		macOS
ProductVersion:		14.1.1
BuildVersion:		23B81

% uname -a
Darwin Matejs-MBP 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000 arm64

And Ubuntu 22.04 running in Multipass on the same laptop:

$ c++ --version
c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ lsb_release --all
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.3 LTS
Release:	22.04
Codename:	jammy

$ uname -a
Linux xts 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct  9 18:01:07 UTC 2 aarch64 aarch64 aarch64 GNU/Linux

@aleks-f
Copy link
Member

aleks-f commented Nov 29, 2023

@matejk please mark as fixed if it is verified

matejk added a commit that referenced this issue Nov 30, 2023
@matejk
Copy link
Contributor

matejk commented Jan 12, 2024

The problem can be reproduced on macOS with CMAKE if:

  • CMAKE_BUILD_TYPE is Release
  • set(CMAKE_CXX_VISIBILITY_PRESET hidden) is set

Poco 1.13 does not build on macOS with the aforementioned changes due to missing *_API directives on some classes and structs.

Poco does not build on Ubuntu Linux 22.04 (gcc 11.4) also with the same CMake settings.

@matejk matejk reopened this Jan 12, 2024
@matejk matejk modified the milestones: Release 1.13.0, Release 1.13.1 Jan 12, 2024
matejk added a commit that referenced this issue Jan 13, 2024
matejk added a commit that referenced this issue Jan 13, 2024
matejk added a commit that referenced this issue Jan 13, 2024
matejk added a commit that referenced this issue Jan 13, 2024
…::column template functions only for supported types. (-fvisibility=hidden) (#4393, #3331)
matejk added a commit that referenced this issue Jan 14, 2024
…::column template functions only for supported types. (-fvisibility=hidden) (#4393, #3331)
matejk added a commit that referenced this issue Jan 15, 2024
matejk added a commit that referenced this issue Jan 17, 2024
…erly (#4394)

* fix(ActiveRecord): missing ActiveRecordLib_API definitions for clang/gcc.

* fix(FPEnvironment): export FPEnvironmentImpl classes (#4393, #3331)

* fix(Crypto): export *Impl classes used from inlines (#4393, #3331)

* fix(Dynamic): explicitly instantiate and export Dynamic::Struct for string and int (-fvisibility=hidden) (#4393, #3331)

* fix(JSON): explicitly instantiate and export SharedPtr for JSON::Array and JSON::Object (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Set symbol visibility to hidden (#4393, #3331)

* enh(configure): user c++17 standard for iphone, Darwin and ARM-Linux.

* fix(UTF): explicitly instantiate and export 16 and 32-bit strings (-fvisibility=hidden) (#4393, #3331)

* fix(RecordSet): make Extraction.h internal and instantiate RecordsSet::column template functions only for supported types. (-fvisibility=hidden) (#4393, #3331)

* fix(UTF): fix explicitly instantiation on Windows (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Add github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(CppParser): Add missing declarations for CppParser_API (#4393, #3331)

* enh(CMake): Enable more options in github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(MongoDB): Add missing MongoDB_API (#4393, #3331)
matejk added a commit that referenced this issue Jan 17, 2024
…erly (#4394)

* fix(ActiveRecord): missing ActiveRecordLib_API definitions for clang/gcc.

* fix(FPEnvironment): export FPEnvironmentImpl classes (#4393, #3331)

* fix(Crypto): export *Impl classes used from inlines (#4393, #3331)

* fix(Dynamic): explicitly instantiate and export Dynamic::Struct for string and int (-fvisibility=hidden) (#4393, #3331)

* fix(JSON): explicitly instantiate and export SharedPtr for JSON::Array and JSON::Object (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Set symbol visibility to hidden (#4393, #3331)

* enh(configure): user c++17 standard for iphone, Darwin and ARM-Linux.

* fix(UTF): explicitly instantiate and export 16 and 32-bit strings (-fvisibility=hidden) (#4393, #3331)

* fix(RecordSet): make Extraction.h internal and instantiate RecordsSet::column template functions only for supported types. (-fvisibility=hidden) (#4393, #3331)

* fix(UTF): fix explicitly instantiation on Windows (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Add github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(CppParser): Add missing declarations for CppParser_API (#4393, #3331)

* enh(CMake): Enable more options in github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(MongoDB): Add missing MongoDB_API (#4393, #3331)
@matejk matejk added the bug label Feb 2, 2024
aleks-f added a commit that referenced this issue Feb 6, 2024
* doc(ReleaseNotes): fix formatting, add PR links

* Incorporated Debian patches (#4380)

* Debian: Use null as device file as console might not be there

* Debian: Add GNU Hurd support

* Debian: Includes not available on Hurd

* Debian: Disable SHA2 test on platforms where it's broken

* Debian: Set POCO_NO_FPENVIRONMENT for armel

---------

Co-authored-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>

* fix(UUID): UUID parser silently ignores too long strings #4375 (#4384)

* fix(Crypto): EVP_CIPHER_CTX_init is incorrectly defined in Envelope.cpp if it is not defined already by OpenSSL. Fixed to properly use EVP_CIPHER_CTX_reset.

* enh(ci): Add macos sanitizers job (#4313)

* enh(ci): macOS thread sanitizer

* enh(ci): macOS sanitize jobs for undefined and address.

* fix(test): lock std:cerr to prevent data race in TCP server tests (reported by clang thread sanitizer) #4313

* fix(test): Use 96-bit IV with aes-256-gcm to fix (#4347):

I/O error: error:1C800066:Provider routines::cipher operation failed

* mingw compile and link improvements (#4019) (#4391)

* fix(platform): MinGW Compile and link errors: undefined reference to `WinMain'

* fix(platform): MinGW compile UUID tests (conflicting UUID defined as GUID in rpcdce.h via windows.h)

* enh(DateTimeParser): option to cleanup input string before parsing (#569).

* fix(CppUnit): do not install #4398

* fix(DataTest): do not install #4398

* chore(SingleSocketPoller): spelling

* fix(MailMessage): Compare lowercase content disposition headers when reading parts (#3650).

* chore(cmake): CppUnit Foundation dependency documentation; fix indentation

* fix(SocketReactorTest): deadlock test intermittently hangs #4400

* gcc/clang (-fvisibility=hidden): corrections to compile and work properly (#4394)

* fix(ActiveRecord): missing ActiveRecordLib_API definitions for clang/gcc.

* fix(FPEnvironment): export FPEnvironmentImpl classes (#4393, #3331)

* fix(Crypto): export *Impl classes used from inlines (#4393, #3331)

* fix(Dynamic): explicitly instantiate and export Dynamic::Struct for string and int (-fvisibility=hidden) (#4393, #3331)

* fix(JSON): explicitly instantiate and export SharedPtr for JSON::Array and JSON::Object (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Set symbol visibility to hidden (#4393, #3331)

* enh(configure): user c++17 standard for iphone, Darwin and ARM-Linux.

* fix(UTF): explicitly instantiate and export 16 and 32-bit strings (-fvisibility=hidden) (#4393, #3331)

* fix(RecordSet): make Extraction.h internal and instantiate RecordsSet::column template functions only for supported types. (-fvisibility=hidden) (#4393, #3331)

* fix(UTF): fix explicitly instantiation on Windows (-fvisibility=hidden) (#4393, #3331)

* enh(CMake): Add github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(CppParser): Add missing declarations for CppParser_API (#4393, #3331)

* enh(CMake): Enable more options in github jobs for macOS with visibility set to hidden (#4393, #3331)

* fix(MongoDB): Add missing MongoDB_API (#4393, #3331)

* Implemented automated network library initialization for Windows MinGW targets (#4402)

* Implemented automated network library initialization for Windows MinGW/GCC targets

* Using POCO_COMPILER_MINGW instead of __GNUC__

---------

Co-authored-by: Jesse Hoogervorst <jesse@deltaxlab.com>

* fix(Thread_POSIX): qnx build error: 'prctl' was not declared in this scope #4404

* fix: NULL pointer strategy when setting rotation never #4411

Regression from 66e93f9.

* fix(progen): add LanguageStandard (stdcpp17, stdc11); regenerate vs170 projects

* Implement GetAdaptersAddresses API (#4419)

* Upgrade from GetAdaptersInfo to GetAdaptersAddresses API. The code has been swapped back to a buffer of bytes because the data structure built by GetAdaptersAddresses is a linked list and the returned size is not a multiple of the IP_ADAPTERS_ADDRESSES struct.

* Adding back Poco/UnWindows.h

* Undoing indents.

* test(ThreadPool): unit test for thread pool shutdown when no worker is running. (#2450)

* enh: #4216: use std::string literals

* enh: #3890: Get rid of SingletonHolder

* enh(File): Linux, macOS: microsecond precision for file times (create and modification time).

* enh(tests): Ability to enable/disable testing of deprecated functionality. (#4425)

* fix(SSLManager): Fixed regression introduced in PR #4103, fixes #4421

* fix(LogFile): Unify flushing behaviour of WIN32 and STD implementation (#2443)

* chore(buildwin): remove old vs versions from build and progen scripts; update documentation

* chore(buildwin): remove leftover closing curly

* enh(SQLite): SQLite FTS5 #4367

* Release 1.13.1: Update release notes, changelog, contributors, version files. (#4440)

* Update CONTRIBUTORS

* fix(CppParser): Documentation generation (some minor fixes, WiP) #4441

* feat(CppParser): C++11 attributes support

* feat(PocoDoc): C++11 attributes support

* chore(doc): Changelog and release notes formatting

* fix(CppParser): parsing of function template parameters and namespace imports

* fix: make headers parseable by CppParser/PocoDoc

* fix(PocoDoc): add -DPOCO_DOC

* fix(PocoDoc): postgres headers not found

* fix(PocoDoc): libpq include path

* fix(XML): #4443: Upgrade libexpat to 2.6.0

* doc: updated changelog

---------

Co-authored-by: Günter Obiltschnig <guenter.obiltschnig@appinf.com>
Co-authored-by: Matej Kenda <matejken@gmail.com>
Co-authored-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
Co-authored-by: Jesse Hoogervorst <hoogervorstjesse@gmail.com>
Co-authored-by: Jesse Hoogervorst <jesse@deltaxlab.com>
Co-authored-by: Aron Budea <aron.budea@collabora.com>
Co-authored-by: Andrew Auclair <andrewauclair@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

6 participants