Software that processes papers in PDF format by calling Grobid's web service and makes a wordcloud for each of them as well as giving a graph indicating the number of figures per article and a list of links found in all of them.
- Papers used for input must have an abstract section or the software will fail.
- Docker must be installed
- Download the Grobid docker image with
docker pull lfoppiano/grobid:0.7.2This build has been developed on Python 3.10 and should work with higher versions.
Python libraries matplotlib and wordcloud must be previously installed.
Dependencies can be found here to use them to build the environment with Conda
You can install Conda to easily install all the dependencies needed on an environment(recommended)
If you don't want to use Conda then skip step 3 from the Instructions segment
- Copy this repo
git clone https://github.com/andriumon/OpenScienceAI.git- Go to the repo and then to the src directory
cd OpenScienceAI/src - Install dependencies or copy the dependencies file to the src directory and use Conda to do it with
conda create -n newenv
conda activate newenv
python3 -m pip install --upgrade pip
pip install -r dependencies.txtNote: If python3 doesn't work, try py
- Create a folder called "pdfs" in the src directory and put inside all the papers you want to process
- Install Grobid's Python Client there
- Run Grobid with Docker
docker run -t --rm -p 8070:8070 lfoppiano/grobid:0.7.2- Run the script
python3 pdfProcessing.pyYou can check the results in the folders "wordclouds", "figures" and "links", which will be created in the directory after you run the script.
Main author and contact: andres.montero.martin@alumnos.upm.es