command-line-interface

[naustica]

• Command-line interface
• Unit tests for cli
• Functions to download and view PDFs

[bganglia]

It looks good to me, but the open command does not work on my system (Ubuntu). xdg-open does work though.

Apparently you can find out the operating system from within python, but I don't know if that is useful.
https://stackoverflow.com/questions/1854/what-os-am-i-running-on/58071295#58071295

I am not sure how to do this in Windows though.

It might also be good to use '.' as a default for the path argument.

[naustica]

thanks, I will fix that. You mean the open command on line 301, right?

[naustica]

Also, is there an alternative way to express urllib.request.urlopen(pdf_link) with the requests library?

[bganglia]

Yes, I guess you could put the response text into a BytesIO object. Something like

from io import BytesIO
response = requests.get("http://google.com")
BytesIO(bytearray(response.text,encoding="utf-8"))

[naustica]

@bganglia Thanks. Did you test the BytesIO variant with your pdfminer?

[bganglia]

@naustica Not yet. Let me try that now.

[bganglia]

@naustica It looks like it works the same as urllib. For example this one works:

>>> minecart.Document(io.BytesIO(bytearray(requests.get("https://www.jns-journal.com/article/S0022-510X(20)30168-4/pdf").text,encoding="utf-8")))
WARNING:root:Cannot locate objid=215
<minecart.miner.Document object at 0x7f1a71833110>

But now that I look at it more closely, sometimes both urllib and requests fail because a link like https://doi.org/10.1016/j.jns.2020.116832 is just some HTML that redirects the browser to a PDF. So I will look for a more robust way of downloading the PDF. Selenium would be overkill, so there must be some simpler way

[naustica]

@bganglia That looks interesting. Let me hear when you find out something new.

[naustica]

@bganglia Can you review my code again? I would then merge the code into the develop branch.

[bganglia]

@naustica I am taking a look at it right now.

If no PDF is available, get_pdf_link returns None, so view_pdf should check for that. However, if you want, we could just make this an issue and address it later.

Everything else works great on Linux. I can try it on Windows too.

[naustica]

@bganglia I would tackle the issue when Im writing the remaining tests for the unpywall class if thats ok for you.

[bganglia]

@naustica Ok, sounds good. I would say that it looks great to merge then