textsum:AssertionError: Empty filelist.

[loretoparisi]

Please let us know which model this issue is about (specify the top-level directory)

textsum

I get this error while doing the training

Exception in thread Thread-152:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Volumes/MacHDD2/Developmemt/ParisiLabs/ML/models/textsum/batch_reader.py", line 135, in _FillInputQueue
    (article, abstract) = input_gen.next()
  File "/Volumes/MacHDD2/Developmemt/ParisiLabs/ML/models/textsum/batch_reader.py", line 244, in _TextGenerator
    e = example_gen.next()
  File "/Volumes/MacHDD2/Developmemt/ParisiLabs/ML/models/textsum/data.py", line 90, in ExampleGen
    assert filelist, 'Empty filelist.'
AssertionError: Empty filelist.

I have my data and vocab and the WORKSPACE in the same dir:

(tensorflow) admin@macbookproloreto:~/Developmemt/ParisiLabs/ML/models/data$ ls -l
total 320
-rw-r--r--  1 admin  staff   33582 30 Ago 17:45 data
-rw-r--r--  1 admin  staff  124934 30 Ago 17:45 vocab

The command launched was

$ bazel-bin/textsum/seq2seq_attention --mode=train --article_key=article --abstract_key=abstract --data_path=data/training-* --vocab_path=data/vocab --log_root=textsum/log_root --train_dir=textsum/log_root/train

[loretoparisi]

I have tried the same on

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.4 LTS
Release:    14.04
Codename:   trusty

My dir structure is

ubuntu@ip-10-169-182-86:~/tensorflow/lyrics$ ls -R
.:
bazel-bin  bazel-genfiles  bazel-lyrics  bazel-out  bazel-testlogs  data  textsum  WORKSPACE

./data:
data  vocab

./textsum:
batch_reader.py   beam_search.py   BUILD  data.py   README.md                    seq2seq_attention_decode.pyc  seq2seq_attention_model.pyc  seq2seq_lib.py
batch_reader.pyc  beam_search.pyc  data   data.pyc  seq2seq_attention_decode.py  seq2seq_attention_model.py    seq2seq_attention.py         seq2seq_lib.pyc

./textsum/data:
data  vocab

I do not have any training- folder in the --data_path path data/training-*.

test-0  training-0  training-1  validation-0 ...(omitted)

So I assume that was the source of this issue.

Command was

$ bazel-bin/textsum/seq2seq_attention   --mode=train   --article_key=article   --abstract_key=abstract   --data_path=data/training-*   --vocab_path=data/vocab   --log_root=textsum/log_root   --train_dir=textsum/log_root/train

[jamcar23]

@loretoparisi you need to give it the correct path to your data. You're telling it to look for data/training-* but you don't have anything that matches that. It looks like you need to use --data_path=data/data

[peterjliu]

Yes thanks @jamcar23, the issue is the toy data provided is data/data. We don't provide the full training data in the repo.

[loretoparisi]

@peterjliu ok thanks, where we get the training data then?

[monajalal]

@panyx0718 @peterjliu considering the fact that this data is not free and apparently there's no other data in this format, would it make sense to use the command below in order to train?

jalal@klein:~/computer_vision/tensorflow/models$ bazel-bin/textsum/seq2seq_attention --mode=train --article_key=article --abstract_key=abstract --data_path=textsum/data/data --vocab_path=textsum/data/vocab --log_root=textsum/log_root --train_dir=textsum/log_root/train

What are other tricks to get data in the data folder?

http://stackoverflow.com/questions/39182142/tensorflow-text-summarization-setup-what-is-a-workspace-file/39184573#39184573