Fix github issue #3269 where the accuracy is wrongly underestimated for binary classification and build issue #2784

[a-dai]

PiperOrigin-RevId: 186265033

[tensorflow-jenkins]

Can one of the admins verify this patch?

[stefan-it]

@a-dai I would like to test the adverserial_text model and followed the installation instructions described in the readme. But using this command:

bazel run data:gen_vocab -- \
    --output_dir=$IMDB_DATA_DIR \
    --dataset=imdb \
    --imdb_input_dir=/tmp/aclImdb \
    --lowercase=False

The following error message occurs:

Extracting Bazel installation...
............
INFO: Analysed target //research/adversarial_text/data:gen_vocab (14 packages loaded).
INFO: Found 1 target...
Target //research/adversarial_text/data:gen_vocab up-to-date:
  bazel-bin/research/adversarial_text/data/gen_vocab
INFO: Elapsed time: 8.194s, Critical Path: 0.03s
INFO: Build completed successfully, 4 total actions

INFO: Running command line: bazel-bin/research/adversarial_text/data/gen_vocab '--output_dir=/tmp/imdb' '--dataset=imdb' '--imdb_input_dir=/tmp/aclImdb' '--lowercase=False'
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/5f97a7a6eda3c5609242fd0fa6b7c9aa/execroot/__main__/bazel-out/k8-fastbuild/bin/research/adversarial_text/data/gen_vocab.runfiles/__main__/research/adversarial_text/data/gen_vocab.py", line 26, in <module>
    from adversarial_text.data import data_utils
ImportError: No module named adversarial_text.data
ERROR: Non-zero return code '1' from command: Process exited with status 1

This was also reported in #2784. How did you solve that issue? I'm using TensorFlow 1.5.

Thanks in advance :)

[a-dai]

I just added a commit to this PR that should fix #2784, can you test it?

[stefan-it]

Thanks for your reply! I used 9ee7363 but I still got the same error message for data:gen_vocab.

[a-dai]

Sorry that was the wrong fix, can you try with this new commit?

[stefan-it]

gen_vocab and gen_data are working now, thanks!

The command:

bazel run :pretrain -- \
    --train_dir=$PRETRAIN_DIR \
    --data_dir=$IMDB_DATA_DIR \
    --vocab_size=86934 \
    --embedding_dims=256 \
    --rnn_cell_size=1024 \
    --num_candidate_samples=1024 \
    --batch_size=256 \
    --learning_rate=0.001 \
    --learning_rate_decay_factor=0.9999 \
    --max_steps=100000 \
    --max_grad_norm=1.0 \
    --num_timesteps=400 \
    --keep_prob_emb=0.5 \
    --normalize_embeddings

returns the following error:

ERROR: /mnt/models/research/adversarial_text/BUILD:63:1: no such package 'adversarial_text/data': BUILD file not found on package path and referenced by '//research/adversarial_text:inputs'
ERROR: Analysis of target '//research/adversarial_text:pretrain' failed; build aborted: Loading failed
INFO: Elapsed time: 0.439s
FAILED: Build did NOT complete successfully (1 packages loaded)
ERROR: Build failed. Not running target

[a-dai]

Can you try running python pretrain.py directly instead of using bazel?

[stefan-it]

No problem, the output then is:

Traceback (most recent call last):
  File "pretrain.py", line 30, in <module>
    from adversarial_text import graphs
ImportError: No module named adversarial_text

[a-dai]

Actually my last commit b555eda should fix that.

[stefan-it]

Using that commit the following error message appears:

Traceback (most recent call last):
  File "pretrain.py", line 30, in <module>
    import graphs
  File "/mnt/models/research/adversarial_text/graphs.py", line 28, in <module>
    import inputs as inputs_lib
  File "/mnt/models/research/adversarial_text/inputs.py", line 27, in <module>
    from data import data_utils
ImportError: No module named data

I solved that by adding an __init__.py file in the data folder. After that pretraining starts :)