Trouble building swivel/fastprep.cc on Mac?

[moscow25]

Hey guys, love the Swivel library. That said, prep.py is too slow (on 1B lines of text dataset), so trying to build the fastprep version.

I get stuck on "rebuild Tensorflow from source" part. It says to build a pip package, but then I get the error:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
cp: bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/tensorflow: No such file or directory
cp: bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/external: No such file or directory

Indeed, those files do not exist. I could not find this error in the other peoples' bugs. A pointer would be appreciated!

Also... if I have vocab and co-occurrences cached from GloVe, can I skip the prep phase?

Thanks!

[moscow25]

To be clear, I have bazel and all the other dependencies installed. I ran the first command, and it returned without error:
bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
...
Target //tensorflow/tools/pip_package:build_pip_package up-to-date:
bazel-bin/tensorflow/tools/pip_package/build_pip_package

And my bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/ exists, just does not include bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/tensorflow

I'm working off of these instructions. Is it possible that something isn't sync'ed?
https://www.tensorflow.org/versions/r0.7/get_started/os_setup.html#installation-for-mac-os-x

[waterson]

Hey there! Apologies for being so slow to respond! :-/

I have not tried building TF from source on Mac; let me take a look at that today and I'll update the issue.

[waterson]

Also... if I have vocab and co-occurrences cached from GloVe, can I skip the prep phase?

And, FWIW, yes, you can definitely do this! Basically you just need re-arrange Glove's co-occurrence matrix into tf.Examples that follow the format that prep.py produces.

[waterson]

I get stuck on "rebuild Tensorflow from source" part. It says to build a pip package, but then I get the error:

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
cp: bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/tensorflow: No such file or directory
cp: bazel-bin/tensorflow/tools/pip_package/build_pip_package.runfiles/external: No such file or directory

Indeed, those files do not exist. I could not find this error in the other peoples' bugs. A pointer would be appreciated!

Okay, I just tried building from source on Mac and... I at least got past this part.

Here is (more or less) what I did. YMMV because it's hard to know how your system is configured. I followed the OS/X "install from source" directions. This involved: installing JDK8, installing homebrew. Then I used homebrew to install bazel and swig, and pip_install to install six, numpy and wheel. Then I pulled the TF source repo, configured it and built the PIP package. Then I built and installed protocol buffers. Finally, I was able to compile fastprep.cc, but failed to link it. I suspect that I need to do some libtool magic to make that work.

brew install bazel swig
sudo easy_install -U six
sudo easy_install -U numpy
sudo easy_install wheel
git clone --recurse-submodules https://github.com/tensorflow/tensorflow
cd tensorflow
./configure
bazel build -c opt //tensorflow/tools/pip_package:build_pip_package
cd google/protobuf
brew install automake libtool
./autogen.sh
./configure --prefix=${HOME} 
make
make install
cd ~/src/swivel/models
make -f fastprep.mk  # this compiles but can't link. so... that's a bug!

LMK if you can get that far; if not, it might make sense to open another issue on the core TF project. In the meantime, there is certainly a real bug here that fastprep.cc can't link.

Also, for what it's worth, note that TF may not support GPUs on OS/X. If that's the case, you may find that Swivel is not going to provide much of a speedup over Glove training.

[waterson]

Okay, by changing

 LDLIBS=-lprotos_all_cc -lprotobuf -lpthread -lm

to

LDLIBS=-lprotos_all_cc.pic -lprotobuf -lpthread -lm

...I can get fastprep.cc to compile and link. So I need to figure out why TF OS/X names its libraries with ".pic" and see if there's a clean way to have fastprep.mk choose the right one.

[moscow25]

Sorry for the late reply @waterson. Your change LDLIBS=-lprotos_all_cc.pic -lprotobuf -lpthread -lm seems to have done it! I have fastprep finally installed. The script runs! Now will have to see if Swivel trains from here :-)

Thanks!

[moscow25]

And thanks @waterson for sharing script to port GloVe co-occurrences to Swivel. Will try that as well. Not sure how I managed to miss notification from your response -- gotta check my GitHub settings. I really appreciate it. Excited to try these comparisons!

[moscow25]

Hi @waterson, unfortunately even though the fastprep compiles, it seems to crash in the co-occurrence phase when writing shards. Have you seen this error before?

For comparison, prep.py works -- albeit very very slowly, so on a 10% sample of this text (which it itself a sample). As long as ulimit -n is set correctly.

I will also separately search for this protobuf error. Thanks!

`$ ./fastprep --output_dir /tmp/swivel --input /Volumes/xxx/rawtweets_10pct.txt --shard_size 16384 --min_count 100 --window_size 15

Computing vocabulary: 100.0% complete...56016451 distinct tokens

Generating Swivel co-occurrence data into /tmp/swivel

Shard size: 16384x16384

Vocab size: 98304

Computing co-occurrences: 100.0% complete...done.

writing marginals...

writing shards...

writing shard 1/36

[libprotobuf FATAL google/protobuf/message_lite.cc:68] CHECK failed: (bytes_produced_by_serialization) == (byte_size_before_serialization): Byte size calculation and serialization were inconsistent. This may indicate a bug in protocol buffers or it may be caused by concurrent modification of the message.

libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: (bytes_produced_by_serialization) == (byte_size_before_serialization): Byte size calculation and serialization were inconsistent. This may indicate a bug in protocol buffers or it may be caused by concurrent modification of the message.

Abort trap: 6
`