Possible bug in Downloading Web Pages with Python

[svmelton]

A reader emailed that she's getting No module named request when trying to use urllib.request She also indicates that she's installed request per https://programminghistorian.org/en/lessons/installing-python-modules-pip#mac-and-linux-instructions and that Atom has been able to run the code so far. (She's using python 3.8 on a Mac.)

Additionally, when I tried to run the code in the lesson, I received an error that the SSL cert couldn't be verified:
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777

(Tagging @acrymble as well, since he just submitted a PR for something else in this lesson.)

[acrymble]

I think this was what @walshbr was doing with this pull request #1785

[walshbr]

Both of those are different errors than the ones for which I just submitted a pull request. I don't have time to take a look in the next week, but I can look at this if no one has gotten to it by then.  But the full stack trace from the person would be helpful for debugging it if you can get it.

[Bibliojo]

Traceback (most recent call last):
File "/Users/bibliojo/Desktop/Documents/PHPython/open-webpage.py", line 3, in
import urllib.request, urllib.parse, urllib.error
ImportError: No module named request

Process returned 1 (0x1) execution time : 0.074 s

[acrymble]

Thanks @Bibliojo. Just making some notes to try to trace this:

https://programminghistorian.org/en/lessons/working-with-web-pages

[acrymble]

@Bibliojo I've just tried this in two different setups on Mac.

The first was in Komodo Edit and I get the same error as you did.
The second is using the Terminal and putting the commands directly in. That worked as intended.

Can you tell me if you've been trying to run this in Komodo Edit using the suggested setup, or if you're using a different approach? It may be the way we've configured Komodo Edit that's the problem.

[Bibliojo]

I am using Atom and not komodo edit. Mostly because i had used atom previously on my work windows machine. i installed atom-run-python and the first several lessons worked. this is where it stopped. both pip and pip3 say that “requests” are installed. i wonder what the communication problem between and and my mac python installation is? thank you for looking into this! jo Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Adam Crymble <notifications@github.com> Sent: Monday, June 15, 2020 4:17:27 AM To: programminghistorian/jekyll <jekyll@noreply.github.com> Cc: Joanne Paterson <jpater22@uwo.ca>; Mention <mention@noreply.github.com> Subject: Re: [programminghistorian/jekyll] Possible bug in Downloading Web Pages with Python (#1810) @Bibliojo<https://github.com/Bibliojo> I've just tried this in two different setups on Mac. The first was in Komodo Edit and I get the same error as you did. The second is using the Terminal and putting the commands directly in. That worked as intended. Can you tell me if you've been trying to run this in Komodo Edit using the suggested setup, or if you're using a different approach? It may be the way we've configured Komodo Edit that's the problem. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1810 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC5LCCQWEWVTW2ZZDWLX4WLRWXKJPANCNFSM4N5QX4KA>.

[walshbr]

@Bibliojo I'm not as familiar with the atom-run-python script - could run these from the same place as you were trying to run the Python script and let us know what the output is (sounds like that is from within Atom)? edited to include a third line

import sys
print(sys.version_info)
print(sys.executable)

[Bibliojo]

open-webpage.py

import urllib.request, urllib.parse, urllib.error

url = 'http://www.oldbaileyonline.org/browse.jsp?id=t17800628-33&div=t17800628-33'

response = urllib.request.urlopen(url)
webContent = response.read()

print(webContent[0:300])

---

Traceback (most recent call last):
File "/Users/bibliojo/Desktop/Documents/PHPython/open-webpage.py", line 3, in
import urllib.request, urllib.parse, urllib.error
ImportError: No module named request

Process returned 1 (0x1) execution time : 0.052 s
Close this window to continue...
bibliojo@Joannes-iMac ~ %

[Bibliojo]

I use F5 to run the script from the saved file, and then the response is in the bash terminal