Attribute Error: 'Document Set' object has no attribute 'title' when building corpus for topic modelling

[SS159]

Would it be possible to add in 'item title' to build.corpus function for topic modelling, as currently, if you use .csv download from Scopus, title and DOI are expressed as 'Item Title' and 'Item DOI', rather than 'Item' and 'DOI'.

Thanks!
S

[stijnh]

Hi SS159. Thanks for using litstudy!

I am not sure if I understand your question. Could you maybe show the code that you are dealing with?

You can load a CSV file using litstudy.load_csv which returns a DocumentSet. That function automatically detects the title as "Item Title" and the DOI as "Item DOI".

You can give this DocumentSet to build_corpus as the first argument to build a corpus.

[SS159]

Hi stijnh,

Thanks for the quick response. Sure, below is the code I'm using. I can't see where the DocumentSet is located/saved (doesn't seem to be in the working directory) - perhaps this is the problem?

[stijnh]

The function refine_scopus returns two DocumentSets: the documents found on Scopus and the document not found on Scopus.

You can use it as follows:

docs_springer, docs_notfound  = litstudy.refine_scopus(docs_springer)

Here docs_springer is the DocumentSet that contains the documents found on Scopus. Give this value to build_corpus.

And docs_notfound is the DocumentSet that contains the documents not found on Scopus. You can ignore this value.

[SS159]

Thanks for getting back to me Stijnh. Unfortunately, I still can't get litstudy to build a corpus from the papers I have downloaded from Scopus, as I am finding the following error

Note that: 1) I am using, for now, only one .csv which is from Scopus and contains 72 papers. That means that I have not merged any .csv files; 2) I have not included the function which separates documents found and not found on Scopus; 3) I note that .csv files downloaded from Scopus are not automatically saved in the format 'title' and 'doi', but rather 'item title' and 'item doi', which may also be why litstudy is unable to find attribute 'title' in the documentset (which, again, in this case is just one document 'docs_springer').

Thanks,

SS159

[SS159]

Hi again, just tried with the .csv from Springer instead, (which also automatically saves as 'Item Title') - can't seem to get it to work with this one either, even after changing the .csv metadata from 'Item Title' to 'Title'

[stijnh]

Hi.

Replace the line

docs_springer = litstudy.refine_scopus(docs_springer)

by

docs_springer, docs_notfound = litstudy.refine_scopus(docs_springer)