In this study I have implemented a framework for systematically studying disagreements among various academic search engines. I performed experiments using 2500 queries over four academic search engines – Google Scholar, Microsoft Academic Search, Semantic Scholar & Scopus. It has been observed that overlap in search result sets of any pair of academic search engines is significantly low and in most of the cases the search result sets are mutually exclusive.
Query Dataset :-
- Topics from ACM Computing Classification Systems https://www.acm.org/publications/class-2012 have been used as queries to ASEs. This system arranges various computer science topics including technology, product, organization names, eminent researchers in computing into a poly-hierarchy ontology. It sufficiently covers broad spectrum of topics in computer science from coarse to fine granularity.
- We also manually collected over 200 keywords from papers published in ACM SIGKDD 2016 conference.
Search Engines :- We queried four ASEs: Google Scholar (GS), Semantic Scholar (SS), Microsoft Academic (MA), and Scopus (SC). Main reason for choosing these ASEs was that they are popular in computer science and engineering domain.
Original Paper :- On Low Overlap among Search Results of Academic Search Engines (http://papers.www2017.com.au.s3-website-ap-southeast-2.amazonaws.com/companion/p823.pdf). Published in World Wide Web of Conference 2017