Cannot run advanced searches using source feature

[oskarryn]

I wish to get all articles from Neural Information Processing Systems 31 (NIPS 2018) sorted by citations.

Google Scholar has an advanced search feature with Return articles published in field, where typing Advances in Neural Information Processing modifies the search query to:
source:Advances source:in source:Neural source:Information source:Processing
I also modified the dates to custom range 2018-2018 and I can correctly see 1020 results returned (there are supposed to be 1011, close enough). These seem to be already somewhat sorted by citations in Google Scholar, but I was still hoping to use your code to have these articles sorted exactly and saved in a csv. Yet, I can only get about 110 results when executing:

python3 sortgs.py --kw "source:Advances source:in source:Neural source:Information source:Processing" --sortby "cit/year" --startyear 2018 --endyear 2018

Any idea if I can achieve this? Thanks!

[WittmannF]

hey @oskarryn ,

100 results is the default. You can specify the parameter --nresults for getting more results. It would look like:

python3 sortgs.py --kw "source:Advances source:in source:Neural source:Information source:Processing" --sortby "cit/year" --startyear 2018 --endyear 2018 --nresult 1020

However, google ends up blocking if a search is repeated too much. I can't guarantee that it is going to work for acquiring 1020 results.

Regarding the advanced search, I think the way you wrote your query was the ideal way. I tried with NIPS 2018 as publisher in advanced search and only got 19 results.

[oskarryn]

Thanks for help. I tried with --nresults 1020 (btw. there is --nresult 1020 in your post), but this doesn't seem to work. Maybe it's the google's scholar cap you mentioned, but I suspect a bug because a limit of 110 searches would be really low.

[WittmannF]

Hey @oskarryn! There was a minor bug in n results. It should be working now. If needed, please reopen the issue