Installation Issues, Hardware Requirements?

[Matrix7689]

Hello,
I am working on Abstractive Text Summarization and I am facing multiple issues such as:

ISSUE #1 - RAM
I was trying to set it up on AWS t2.micro, but spaCy couldn't install due to low RAM (1 GB). Upgraded to 2 GB RAM and then spaCy installed successfully.
Later I was trying to run the Stanford-CoreNLP Server to test it, it did run but it couldn't execute any sentences and crashed with this error:
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000075072e000, 661209088, 0) failed; error='Not enough space' (errno=12)
Which I am guessing is a RAM error...
So in order to run this model, what amount of RAM is required?
What OS and hardware specifications (Graphic Card, etc.) would you recommend?

ISSUE #2 - Dataset
When I was setting up the "Bytecup2018" dataset, on the following command,
python3 tokenize.py --input bytecup.corpus.train.1.txt --output new1.txt
I got this error:
ImportError: cannot import name 'StanfordCoreNLP'
Which means that StanfordCoreNLP wasn't installed properly. Do we have to use the pywrapper version of StanfordCoreNLP? Because the link that you have given consists of java files. How should I go about it?

In order to run the model, do I have to install all three datasets- CNN/Dailymail, Newsroom and Bytecup?

ISSUE #3 - Pre-trained Model
Where do I place the pre-trained model?

[ananroha]

Hi Matrix

ISSUE #3 - Pre-trained Model
Where do I place the pre-trained model?

You have to save the Model in the following format:

Project
------LeafNATS-Master
-------------LeafNATS
-------------etc...
------Model

I have a question

Can we use leafNATS if I dont have a GPU?
Because I run into this error

File "C:\Users\deranan1\AppData\Local\Continuum\anaconda3\lib\site-packages\torch\cuda_init_.py", line 75, in _check_driver
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

[tshi04]

Hi Matrix7689,

ISSUE #1: I haven't tested how much RAM needed to run this code. But I do run it on my mac-pro without GPU. The codes are managed and tested on Ubuntu environment. To test the pre-trained model, we used SpaCy as a tokenizer.

ISSUE #2: Yes.
from pycorenlp import StanfordCoreNLP
nlp = StanfordCoreNLP('http://localhost:9000')
You still need to install StanfordCoreNLP when you use the pycorenlp wrapper.

No, you don't have to install data to run the model. Here is the link to our pre-processed cnn/dm bytecup dataset.
https://drive.google.com/open?id=1KEYj5OtNIe4XkO_NxDzs808QC9zkznIm
https://drive.google.com/open?id=1RknrtkbUIkpilJUFGiHN4Oi85xhnb3vV
You only need to

1. un-comment
   from LeafNATS.playground.summarization.pointer_generator_network.main import *
   and comment other stuff.
2. run
   python3 run.py --data_dir your_data_dir to train your model.

ISSUE #3

1. un-comment
   from LeafNATS.playground.summarization.headline2_summary2_app.main_app import *
2. Download pretrained model.
3. Direct your program to app_model_dir and app_data_dir
   see
   parser.add_argument('--app_model_dir', default='../model/', help='directory that stores models.')
   parser.add_argument('--app_data_dir', default='/var/www/html/leafNATS/tmp', help='directory that stores data.')

[tshi04]

ISSUE #3

Please also check this link
https://github.com/tshi04/LeafNATS/tree/master/LeafNATS/playground/summarization/headline2_summary2_app

[tshi04]

Can we use leafNATS if I dont have a GPU?

Yes. change device to cpu.
parser.add_argument('--device', default=torch.device("cuda:0"), help='device')

[ananroha]

Hi Tshi

Yes it ran on the CPU but with the code -->
parser.add_argument('--device', default=torch.device("cpu"), help='device')