PARQUET-416: C++11 compilation, code reorg, libparquet and installation targets

[wesm]

Reorganize code into a top level src/parquet directly, add a libparquet shared library, and add install targets for libparquet and its header files. Add cpplint script and make lint target for code linting.

Replaces earlier PR #13

[wesm]

OS X is failing with a homebrew error on brew install thrift; trying work around for now, if not we can disable OS X for the time being.

[wesm]

I disabled OS X in the Travis CI setup for now; the .travis.yml will need some more twiddling which can go in a separate patch. For some reason brew install thrift was failing with a checksum error

[nongli]

I tried this patch on osx by cloning and following the build steps in the readme. It worked on master but not this patch.

cmake .
...
Make Warning (dev):
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   parquet

This warning is for project developers.  Use -Wno-dev to suppress it.

make 
[ 50%] Linking CXX shared library build/debug/libparquet.dylib
Undefined symbols for architecture x86_64:
  "snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)", referenced from:
      parquet_cpp::SnappyCodec::Compress(int, unsigned char const*, int, unsigned char*) in libparquet_compression.a(snappy-codec.cc.o)
  "snappy::RawUncompress(char const*, unsigned long, char*)", referenced from:
      parquet_cpp::SnappyCodec::Decompress(int, unsigned char const*, int, unsigned char*) in libparquet_compression.a(snappy-codec.cc.o)
  "snappy::MaxCompressedLength(unsigned long)", referenced from:
      parquet_cpp::SnappyCodec::MaxCompressedLen(int, unsigned char const*) in libparquet_compression.a(snappy-codec.cc.o)
  "typeinfo for apache::thrift::transport::TMemoryBuffer", referenced from:
      boost::shared_ptr<apache::thrift::transport::TMemoryBuffer> boost::dynamic_pointer_cast<apache::thrift::transport::TMemoryBuffer, apache::thrift::transport::TTransport>(boost::shared_ptr<apache::thrift::transport::TTransport> const&) in parquet.cc.o
  "typeinfo for apache::thrift::transport::TTransportException", referenced from:
      apache::thrift::transport::TMemoryBuffer::TMemoryBuffer(unsigned char*, unsigned int, apache::thrift::transport::TMemoryBuffer::MemoryPolicy) in parquet.cc.o
      apache::thrift::transport::TTransport::open() in parquet.cc.o
      apache::thrift::transport::TTransport::close() in parquet.cc.o
      apache::thrift::transport::TTransport::read_virt(unsigned char*, unsigned int) in parquet.cc.o
      apache::thrift::transport::TTransport::write_virt(unsigned char const*, unsigned int) in parquet.cc.o
      apache::thrift::transport::TTransport::consume_virt(unsigned int) in parquet.cc.o
      unsigned int apache::thrift::transport::readAll<apache::thrift::transport::TTransport>(apache::thrift::transport::TTransport&, unsigned char*, unsigned int) in parquet.cc.o
      ...
  "vtable for apache::thrift::transport::TMemoryBuffer", referenced from:
      apache::thrift::transport::TMemoryBuffer::TMemoryBuffer(unsigned char*, unsigned int, apache::thrift::transport::TMemoryBuffer::MemoryPolicy) in parquet.cc.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
  "vtable for apache::thrift::transport::TTransportException", referenced from:
      apache::thrift::transport::TTransportException::TTransportException(apache::thrift::transport::TTransportException::TTransportExceptionType, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in parquet.cc.o
  NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture x86_64

[nongli]

why doesn't this have more things in it? e.g. snappy

[nongli]

Ok, If i add snappystatic and thriftstatic, it compiles.

[wesm]

Dynamic linking on OS X is a nightmare. With the fix I put in ./debug/decode-benchmark works and dynamically links to libparquet.dylib properly.

[wesm]

As soon as we get the build environment question sorted out it would be good to use the same linkage as libparquet for the dependencies that have both static and shared libraries available.

[nongli]

Cool, I tried the latest version in your branch and it builds.

[nongli]

@wesm can you fix the build issue. I think you just need to add -fPIC

[nongli]

This looks good to me. Can you file a jira and prefix the commit so the title is
PARQUET-XXX: description?

[wesm]

Ready for cherry-picking. I updated the PARQUET-416 title and description. Thanks!

[nongli]

@wesm is disabling osx still necessary?

[wesm]

I can try the build again to see if the Homebrew thrift error cleared. I also could not get Thrift to build from source on Travis yesterday

[nongli]

Nvm. I got confused. It's not the osx issues that that been fixed.

Thanks. I'll merge this later today.

[grundprinzip]

can you wrap this in if (APPLE) please?

[wesm]

@grundprinzip sure; I'm seeing if OS X travis build will pass now and then will squash all these again

[kalaxy]

Let me know when this is no longer in flux and I'lll rebase on top of it again with my pull request.

[wesm]

@kalaxy I got gcc-4.9 working and got the OS X build passing again (the magic trick was to call brew update before using brew install >:|). @nongli this should be good to cherry pick if the build is passing

[nongli]

@wesm Thanks! Can you not squash in the future? It makes it hard to review the deltas and the merge tool we use will squash them.

[wesm]

Sure, no problem.