Building within termux fails with 'incomplete type CBlock'

[esotericnonsense]

Describe the issue

Building within Termux fails in validationinterface. The error is below.

Can you reliably reproduce the issue?

If so, please list the steps to reproduce below:

1. Install the required dependencies in Termux using the package manager
2. Compile libevent from source to get libevent_pthreads
3. Wrangle a bit with chroots and so on to get autogen/configure to run through
4. make

Expected behaviour

A copy of bitcoind to appear for me to abuse my smartphone with. Obviously not a 'supported' configuration, but it'd be a fun toy to play with.

Actual behaviour

... omitted ...
  CXX      libbitcoin_server_a-validationinterface.o                                                                  
In file included from validationinterface.cpp:6:                                                                      
In file included from ./validationinterface.h:9:                                                                      
In file included from /usr/bin/../include/c++/v1/memory:599:                                                          
/usr/bin/../include/c++/v1/type_traits:1334:59: error: incomplete type 'CBlock' used in type trait expression         
    : public integral_constant<bool, __is_base_of(_Bp, _Dp)> {};                                                      
                                                          ^                                                           
/usr/bin/../include/c++/v1/type_traits:543:37: note: in instantiation of template class                               
      'std::__ndk1::is_base_of<std::__ndk1::allocator_arg_t, CBlock>' requested here                                  
struct __lazy_and : __lazy_and_impl<_P1::type::value, _Pr...> {};                                                     
                                    ^                                                                                 
/usr/bin/../include/c++/v1/tuple:664:9: note: in instantiation of template class                                      
      'std::__ndk1::__lazy_and<std::__ndk1::is_base_of<std::__ndk1::allocator_arg_t, CBlock>,                         
      std::__ndk1::__lazy_all<std::__ndk1::__dependent_type<std::__ndk1::is_default_constructible<const CBlock &>,    
      true>, std::__ndk1::__dependent_type<std::__ndk1::is_default_constructible<const CValidationState &>, true> > >'
      requested here                                                                                                  
        __lazy_and<                                                                                                   
        ^                                                                                                             
/usr/bin/../include/c++/v1/tuple:670:5: note: in instantiation of default argument for 'tuple<CBlock,                 
      CValidationState, true>' required here                                                                          
    tuple(_AllocArgT, _Alloc const& __a)                                                                              
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                                                                              
/usr/include/boost/signals2/detail/variadic_slot_invoker.hpp:128:49: note: while substituting deduced template        
      arguments into function template 'tuple' [with _AllocArgT = CBlock, _Alloc = CValidationState, _Dummy =         
      (no value), $3 = (no value)]                                                                                    
        variadic_slot_invoker(Args & ... args): _args(args...)                                                        
                                                ^                                                                     
/usr/include/boost/signals2/detail/signal_template.hpp:239:34: note: in instantiation of member function              
      'boost::signals2::detail::variadic_slot_invoker<boost::signals2::detail::void_type, const CBlock &, const       
      CValidationState &>::variadic_slot_invoker' requested here                                                      
          slot_invoker invoker = slot_invoker(BOOST_SIGNALS2_SIGNATURE_ARG_NAMES(BOOST_SIGNALS2_NUM_ARGS));           
                                 ^                                                                                    
/usr/include/boost/signals2/detail/signal_template.hpp:722:16: note: in instantiation of member function              
      'boost::signals2::detail::signal_impl<void (const CBlock &, const CValidationState &),                          
      boost::signals2::optional_last_value<void>, int, std::__ndk1::less<int>, boost::function<void (const CBlock &,  
      const CValidationState &)>, boost::function<void (const boost::signals2::connection &, const CBlock &, const    
      CValidationState &)>, boost::signals2::mutex>::operator()' requested here                                       
        return (*_pimpl)(BOOST_SIGNALS2_SIGNATURE_ARG_NAMES(BOOST_SIGNALS2_NUM_ARGS));                                
               ^                                                                                                      
validationinterface.cpp:121:30: note: in instantiation of member function 'boost::signals2::signal<void (const CBlock 
      &, const CValidationState &), boost::signals2::optional_last_value<void>, int, std::__ndk1::less<int>,          
      boost::function<void (const CBlock &, const CValidationState &)>, boost::function<void (const                   
      boost::signals2::connection &, const CBlock &, const CValidationState &)>, boost::signals2::mutex>::operator()' 
      requested here                                                                                                  
    m_internals->BlockChecked(block, state);                                                                          
                             ^                                                                                        
./validationinterface.h:13:7: note: forward declaration of 'CBlock'                                                   
class CBlock;                                                                                                         
      ^                                                                                                               
1 error generated.                                                                                                    
make[2]: *** [Makefile:5891: libbitcoin_server_a-validationinterface.o] Error 1                                       
make[2]: *** Waiting for unfinished jobs....               
make[2]: Leaving directory '/home/src/bitcoin-0.15.0/src'  
make[1]: *** [Makefile:9277: all-recursive] Error 1        
make[1]: Leaving directory '/home/src/bitcoin-0.15.0/src'  
make: *** [Makefile:739: all-recursive] Error 1            

What version of bitcoin-core are you using?

v0.15.0.1, signed source package.

Other potentially relevant information

Termux doesn't ship g++ and hasn't for a while now.

$ clang -v                            
clang version 5.0.0 (tags/RELEASE_500/final)               
Target: aarch64--linux-android                             
Thread model: posix          
InstalledDir: /data/data/com.termux/files/usr/bin

[esotericnonsense]

May have fixed this by avoiding the forward declaration(?). Will update if the build finishes.

[esotericnonsense]

Fix(es):

add #include "primitives/block.h" in validationinterface.h
I think this is a specific android bug. Tipped off by android/ndk#262.

hardcode OS_ANDROID in src/leveldb/port/port_posix.h (f{read,write,flush}_unlocked are not present)

test_bitcoin doesn't work because the folders it wants to use are not present. Can probably work around this.

Syncing up from a LAN node, block 40,000 so far.

[TheBlueMatt]

In the error message it seems like it may be possible to resolve the issue by including "primitives/block" in validationinterface.cpp instead of .h. Does that still work?

[laanwj]

Ha, termux - I'm also interested in having this work, would put it within reach to hack on bitcoin core from my phone. Back in the day I stumbled on some pretty obscure build system issues. It's promising that you get this far.

test_bitcoin doesn't work because the folders it wants to use are not present. Can probably work around this.

Does it hardcode any (global) folder names?

[esotericnonsense]

I have uploaded two relevant files that allow bitcoind to compile with disablewallet on Termux using termux-chroot. It passes test_bitcoin, I haven't tried functional tests yet.

https://esotericnonsense.com/tmp/bitcoin.ff4cd6075b12fb32b9a906deea3ed033e3f9560a.diff.txt is a patch on master (base commit in filename).

Hack out boost::filesystem::temp_directory_path() (it returns an inaccessible directory on unrooted Android).
Force OS_ANDROID to be true for leveldb to compile (should be fixable in build system if we can detect android/termux?).
Fix forward declaration issue on Android NDK.

https://esotericnonsense.com/tmp/libevent.8b0aa7b36a3250fad4953f194c8a94ab25032583.diff.txt is a patch on libevent master (base commit in filename).

The libevent one is more of a termux issue than anything to do with us (we need libevent-pthreads which isn't in the termux package). I don't think we use the removed arc4random function (which doesn't exist on Android anyway).

They are pretty lazy patches which basically hardcode stuff to get it to work. If more than a few are interested (?) I could look into the build system at some point.

I don't have a full list of dependencies due to history issues on the system, but I think this should cover most of it:

apt install coreutils
apt install rsync
apt install python
apt install git
apt install binutils
apt install autoconf
apt install automake 
apt install libtool
apt install sed
apt install grep 
apt install util-linux 
apt install proot  # For termux-chroot (needed to set up environment properly)
apt install libllvm
apt install clang
apt install boost-dev pkg-config 
apt install libevent-dev openssl-dev 

Moto G first gen takes approx 15sec to process a block at the tip on mainnet. Not too shabby.

[TheBlueMatt]

@esotericnonsense can you test moving the extra validationinterface include to the cpp file instead of the h file?

…

On September 30, 2017 1:14:07 AM EDT, Daniel Edgecumbe ***@***.***> wrote: I have uploaded two relevant files that allow bitcoind to compile with disablewallet on Termux using termux-chroot. It passes test_bitcoin, I haven't tried functional tests yet. https://esotericnonsense.com/tmp/bitcoin.ff4cd6075b12fb32b9a906deea3ed033e3f9560a.diff.txt is a patch on master (base commit in filename). Hack out boost::filesystem::temp_directory_path() (it returns an inaccessible directory on unrooted Android). Force OS_ANDROID to be true for leveldb to compile (should be fixable in build system if we can detect android/termux?). Fix forward declaration issue on Android NDK. https://esotericnonsense.com/tmp/libevent.8b0aa7b36a3250fad4953f194c8a94ab25032583.diff.txt is a patch on libevent master (base commit in filename). The libevent one is more of a termux issue than anything to do with us (we need libevent-pthreads which isn't in the termux package). I don't think we use the removed arc4random function (which doesn't exist on Android anyway). They are pretty lazy patches which basically hardcode stuff to get it to work. If anyone other than me is that interested (?) I could look into the build system at some point. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #11388 (comment)

[esotericnonsense]

@TheBlueMatt apologies, I think I misread your comment earlier.
That does appear to work - I can try a make clean && make as well (the build process is pretty slow on the device I'm using).

[TheBlueMatt]

OK, well at least that we can fix without much overhead, see 11440.

[fanquake]

Fixed in #11440.