Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix simplifications #140

Merged
merged 12 commits into from Aug 13, 2019

Conversation

@pmconrad
Copy link

commented Jun 24, 2019

@pmconrad pmconrad referenced this pull request Jun 24, 2019
0 of 3 tasks complete
@jmjatlanta

This comment has been minimized.

Copy link

commented Jul 16, 2019

on macOS:

[71%] Linking CXX static library libfc.a
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libfc.a(non_preemptable_scope_check.cpp.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libfc.a(non_preemptable_scope_check.cpp.o) has no symbols
Copying OpenSSL/ssl/openssl.cnf into target directory.
[ 71%] Built target fc
Scanning dependencies of target ecc_test
Scanning dependencies of target all_tests
[ 72%] Building CXX object tests/CMakeFiles/ecc_test.dir/crypto/ecc_test.cpp.o
[ 72%] Building CXX object tests/CMakeFiles/all_tests.dir/all_tests.cpp.o
[ 73%] Linking CXX executable ecc_test
[ 74%] Building CXX object tests/CMakeFiles/all_tests.dir/compress/compress.cpp.o
Undefined symbols for architecture x86_64:
  "typeinfo for unsigned __int128", referenced from:
      fc::from_variant(fc::variant const&, unsigned __int128&, unsigned int) in libfc.a(variant.cpp.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [tests/ecc_test] Error 1
make[1]: *** [tests/CMakeFiles/ecc_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
@jmjatlanta

This comment has been minimized.

Copy link

commented Jul 16, 2019

on Windows 10

C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(75): error C2440: 'type cast': cannot convert from 'std::_Array_iterator<_Ty,33>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj
]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(75): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(75): error C2660: 'secp256k1_ec_pubkey_create': function does not take 4 arguments [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
  C:\Development\cpp\bitshares-fc-pmconrad\vendor\secp256k1-zkp\include\secp256k1.h(277): note: see declaration of 'secp256k1_ec_pubkey_create'
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(96): error C2440: 'type cast': cannot convert from 'std::_Array_iterator<_Ty,65>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj
]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(96): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_impl_priv.cpp(96): error C2660: 'secp256k1_ecdsa_sign_compact': function does not take 6 arguments [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
  C:\Development\cpp\bitshares-fc-pmconrad\vendor\secp256k1-zkp\include\secp256k1.h(193): note: see declaration of 'secp256k1_ecdsa_sign_compact'
  elliptic_secp256k1.cpp
C:\Development\cpp\bitshares-fc-pmconrad\include\fc/variant.hpp(707): warning C4101: 'e': unreferenced local variable [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(73): error C2664: 'int secp256k1_ec_pubkey_tweak_mul(const secp256k1_context_t *,unsigned char *,int,const unsigned char *)': cannot convert argument 2 fro
m 'std::_Array_iterator<_Ty,33>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(73): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(74): error C2440: 'type cast': cannot convert from 'std::_Array_iterator<_Ty,33>' to 'char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(74): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(107): error C2664: 'void *memcpy(void *,const void *,size_t)': cannot convert argument 1 from 'std::_Array_iterator<_Ty,33>' to 'void *' [C:\Development\cp
p\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(107): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(108): error C2664: 'int secp256k1_ec_pubkey_tweak_add(const secp256k1_context_t *,unsigned char *,int,const unsigned char *)': cannot convert argument 2 fr
om 'std::_Array_iterator<_Ty,33>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(108): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(128): warning C4267: 'initializing': conversion from 'size_t' to 'unsigned int', possible loss of data [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj
]
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(129): error C2664: 'void *memcpy(void *,const void *,size_t)': cannot convert argument 1 from 'std::_Array_iterator<_Ty,65>' to 'void *' [C:\Development\cp
p\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(129): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(130): error C2664: 'int secp256k1_ec_pubkey_decompress(const secp256k1_context_t *,unsigned char *,int *)': cannot convert argument 2 from 'std::_Array_ite
rator<_Ty,65>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(130): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(145): error C2440: 'initializing': cannot convert from 'std::_Array_iterator<_Ty,33>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcx
proj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(145): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(168): error C2664: 'int secp256k1_ecdsa_recover_compact(const secp256k1_context_t *,const unsigned char *,const unsigned char *,unsigned char *,int *,int,i
nt)': cannot convert argument 3 from 'std::_Array_const_iterator<_Ty,65>' to 'const unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(168): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(181): error C2664: 'H fc::hmac<H>::digest(const char *,uint32_t,const char *,uint32_t)': cannot convert argument 3 from 'std::_Array_const_iterator<_Ty,37>
' to 'const char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              H=fc::sha512
          ]
          and
          [
              _Ty=char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(181): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called
C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(184): error C2664: 'int secp256k1_ec_pubkey_tweak_add(const secp256k1_context_t *,unsigned char *,int,const unsigned char *)': cannot convert argument 2 fr
om 'std::_Array_iterator<_Ty,33>' to 'unsigned char *' [C:\Development\cpp\bitshares-fc-pmconrad\fc.vcxproj]
          with
          [
              _Ty=unsigned char
          ]
  C:\Development\cpp\bitshares-fc-pmconrad\src\crypto\elliptic_secp256k1.cpp(184): note: No user-defined-conversion operator available that can perform this conversion, or the operator cannot be called

@pmconrad pmconrad force-pushed the pmconrad:1584_more_simplification branch from b3904ad to ce79181 Jul 26, 2019

@pmconrad

This comment has been minimized.

Copy link
Author

commented Jul 26, 2019

Fixed the windows issues and rebased.
Not sure what the mac problem is, the compiler accepts it during compilation but fails while linking???

@pmconrad pmconrad referenced this pull request Jul 26, 2019
@pmconrad

This comment has been minimized.

Copy link
Author

commented Jul 26, 2019

Hm, found this similar problem from a FreeBSD user: https://bugs.llvm.org/show_bug.cgi?id=26156
According to this, it's a problem with libcxx not with the compiler. I guess that means it cannot be made to work on Mac, so we have to fall back to boost there?

@abitmore

This comment has been minimized.

Copy link
Member

commented Jul 29, 2019

Looks good to me so far. Waiting for @jmjatlanta's input.

@jmjatlanta

This comment has been minimized.

Copy link

commented Jul 29, 2019

On macOS:

[ 27%] Built target project_editline
Scanning dependencies of target ecc_test
[ 27%] Building CXX object libraries/fc/tests/CMakeFiles/ecc_test.dir/crypto/ecc_test.cpp.o
[ 27%] Linking CXX executable ecc_test
Undefined symbols for architecture x86_64:
  "typeinfo for unsigned __int128", referenced from:
      fc::from_variant(fc::variant const&, unsigned __int128&, unsigned int) in libfc.a(variant.cpp.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libraries/fc/tests/ecc_test] Error 1
make[1]: *** [libraries/fc/tests/CMakeFiles/ecc_test.dir/all] Error 2
make: *** [all] Error 2

Results from Windows:
https://pastebin.com/NG4q4wU9

@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 2, 2019

More fixes here.
The windows error messages indicate that your core version is not the latest of bitshares/bitshares-core#1789 .

Still unsure how to handle the __int128 problem on Mac. I'd like to preserve it generally for the gitian-osx build. How to distinguish between gitian and native?

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 2, 2019

For Windows, I cloned pmconrad/bitshares-core and checked out 1584_simplification. I then changed the origin of libraries/fc to pmconrad/fc, and checked out 1584_more_simplifications.

The results are here: https://gist.github.com/jmjatlanta/1aeeabe9ca71e6923daeb1ee87aea68a

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 2, 2019

May be unrelated, but it says __int128 and mac: EOSIO/eos#250

@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 5, 2019

May be unrelated, but it says __int128 and mac

Thanks. It points at the root of the problem. Will try a workaround.

@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 5, 2019

Please try again

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 5, 2019

May be a dirty environment, but here is the first error:

[ 37%] Building CXX object libraries/protocol/CMakeFiles/graphene_protocol.dir/account.cpp.o
In file included from /Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/protocol/account.cpp:24:
In file included from /Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/protocol/include/graphene/protocol/account.hpp:26:
In file included from /Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/protocol/include/graphene/protocol/base.hpp:27:
In file included from /Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/protocol/include/graphene/protocol/types.hpp:60:
/Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/protocol/include/graphene/protocol/object_id.hpp:171:17: error: no template
      named 'true_type' in namespace 'fc'; did you mean 'enum_type'?
    typedef fc::true_type  is_defined;
            ~~~~^~~~~~~~~
                enum_type
/Users/JohnJones/Development/cpp/bitshares-core-1584/libraries/fc/include/fc/io/enum_type.hpp:9:9: note: 'enum_type' declared here
  class enum_type
        ^
@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 6, 2019

That should be std::true_type, please check that core is on 1584_simplifications branch.

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 7, 2019

Windows: It compiled
Mac: I wiped my environment and tried to compile, but now get an issue with pthread

-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- The C compiler identification is AppleClang 10.0.1.10010046
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Warning (dev) in CMakeLists.txt:
  A logical block opening on the line

    /Users/JohnJones/Development/cpp/bitshares-core-1584/CMakeLists.txt:15 (if)

  closes on the line

    /Users/JohnJones/Development/cpp/bitshares-core-1584/CMakeLists.txt:19 (endif)

  with mis-matching arguments.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Performing Test HAVE_RELROFULL
-- Performing Test HAVE_RELROFULL - Failed
-- Performing Test HAVE_RELROPARTIAL
-- Performing Test HAVE_RELROPARTIAL - Failed
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Could NOT find Threads (missing: Threads_FOUND) 
CMake Error at /Applications/CMake.app/Contents/share/cmake-3.14/Modules/FindBoost.cmake:2132 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.69.0

  Boost include path:
  /Users/JohnJones/Development/cpp/boost169/include/boost-1_69

  Could not find the following static Boost libraries:

          boost_thread (missing dependency: Threads)

  Some (but not all) of the required Boost libraries were found.  You may
  need to install these additional Boost libraries.  Alternatively, set
  BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
  to the location of Boost.
Call Stack (most recent call first):
  CMakeLists.txt:140 (FIND_PACKAGE)
@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 7, 2019

Thanks!
That sounds like it might be a result of #121 not this PR. Can you verify this? Perhaps enable the old FindBoost included here instead of the system one?

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 7, 2019

I found a StackOverflow piece that fixed the cmake/mac/pthread issue. But now I am investigating the item below (It's back, or it never left):

[ 69%] Linking CXX executable embed_genesis
Undefined symbols for architecture x86_64:
  "typeinfo for unsigned __int128", referenced from:
      void boost::conversion::detail::throw_bad_cast<unsigned __int128, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >() in libgraphene_app.a(util.cpp.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libraries/egenesis/embed_genesis] Error 1

Note: The __int128 issue here is in core, whereas earlier it was fc.

Update: after refreshing my codebase, the __int128 issue in fc was back, so evidently my cmake addition didn't fix the isue. I will try to revert the pr #121 changes.

Update 2: I was able to get everything to compile once. Something is unstable, it may be my environment again. I will clone again, make the change to CMakeLists.txt, and see what happens.

@pmconrad

This comment has been minimized.

Copy link
Author

commented Aug 8, 2019

Replaced lexical_cast on uint128 with fc::variant in core - bitshares/bitshares-core#1789

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 9, 2019

The following is what was added to CMakeLists.txt in core. It was added before the find_boost call:

IF(APPLE)
   set(CMAKE_THREAD_LIBS_INIT "-lpthread")
   set(CMAKE_HAVE_THREADS_LIBRARY 1)
   set(CMAKE_USE_WIN32_THREADS_INIT 0)
   set(CMAKE_USE_PTHREADS_INIT 1)
   set(THREADS_PREFER_PTHREAD_FLAG ON)
ENDIF(APPLE)

I highly doubt this was caused by this PR, but will test on Mac with [core develop] and [fc master] to verify. I will then create a PR in core for this fix.

@pmconrad pmconrad marked this pull request as ready for review Aug 9, 2019

@jmjatlanta

This comment has been minimized.

Copy link

commented Aug 9, 2019

With the unusable state of the development branch on mac, it is difficult to test a PR of my CMakeLists.txt fix without your branch. You could include it as part of this PR, or I can wait until this is merged in and do it as a separate PR.

Regardless, I'll perform a review and test on all platforms again, but with my fix when testing with macOS.

@jmjatlanta
Copy link

left a comment

Looks good. One small cmake warning.

@@ -7,7 +7,7 @@ PROJECT( fc )
set( CMAKE_CXX_STANDARD 14 )
SET( CMAKE_CXX_STANDARD_REQUIRED ON )

if( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU" )
if( "${CMAKE_CXX_COMPILER_ID}" MATCHES "^(Apple)?Clang|GNU$" )

This comment has been minimized.

Copy link
@jmjatlanta

jmjatlanta Aug 9, 2019

CMake Warning (dev) in CMakeLists.txt:
  A logical block opening on the line

    /home/jmjatlanta/Development/cpp/bitshares-fc-1584/CMakeLists.txt:10 (if)

  closes on the line

    /home/jmjatlanta/Development/cpp/bitshares-fc-1584/CMakeLists.txt:14 (endif)

  with mis-matching arguments.

Perhaps C-n-P the "if" into the "else" and "endif". Tested with cmake 3.15.1.

This comment has been minimized.

Copy link
@pmconrad

pmconrad Aug 11, 2019

Author

Done, also in core

@jmjatlanta jmjatlanta merged commit 82af8b5 into bitshares:master Aug 13, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@abitmore abitmore added this to the core release 3.3.0 milestone Aug 21, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.