Python lesson sequence and urllib.request output bug

[ElectronGM]

Reporting a problem with a lesson, please try to provide us with the following information:

• The full title of the lesson: ...\python-lessons9\html-to-pretty-print.py
• The system you are using (Mac, Linux, Windows): Windows
• Version numbers of the relevant software you are using: Python 3.8.2150.0 (64bit)
• The exact steps you took that caused the problem: For Lesson 9, I did the exercise from the zip archive after I had managed to recreate all the previous exercises on my own and I simply couldn't get on with Lesson 9. Unfortunately, the model solution also does not help in the current Python version.
  Command output (in case that the image cannot be seen:
  Traceback (most recent call last):
  File "html-to-pretty-print.py", line 8, in
  text = obo.webPageToText(url)
  File "C:\Gm\GmPython\python-lessons9\python-lessons9\obo.py", line 112, in webPageToText
  text = stripTags(html).lower()
  File "C:\Gm\GmPython\python-lessons9\python-lessons9\obo.py", line 57, in stripTags
  startLoc = pageContents.find("

  ")
  TypeError: argument should be integer or bytes-like object, not 'str'
  

[svmelton]

Thank you, @ElectronGM! Is anyone from @programminghistorian/technical-team and/or @programminghistorian/english-team available to take a look at this? I'm wondering if it's python3 related. (I just received a similar note about a different lesson in my personal email.)

[ElectronGM]

The lessons are provided from https://programminghistorian.org/en/lessons/introduction-and-installation. <https://programminghistorian.org/en/lessons/introduction-and-installation> In Lesson 9 a file will be offered for download as zip on page https://programminghistorian.org/en/lessons/output-keywords-in-context-in-html-file Greetings - ElectronGM Am 17.05.2020 um 15:24 schrieb Brandon Walsh:

…

@ElectronGM <https://github.com/ElectronGM> what is the url for the lesson you're trying to follow? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1768 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMRJ5WUZHW2TQ7A3UMJAOOTRR7QRTANCNFSM4M5IBOLA>.

[walshbr]

Thanks for sharing @ElectronGM -

As @svmelton suggested, this lesson was recently updated to Python. It looks like the process was tested thoroughly for that one section you're working with. The code expects that you'll be working with a string, but the requests .read() function returns a bytes object I believe. You should be able to get it working by adding a single line to the stripTags function to change pageContents to a string like the code wants. Modify the striptags file

def stripTags(pageContents):
    pageContents = str(pageContents) # this is the new line!
    startLoc = pageContents.find("<p>")
    endLoc = pageContents.rfind("<br/>")

    pageContents = pageContents[startLoc:endLoc]

    inside = 0
    text = ''

    for char in pageContents:
        if char == '<':
            inside = 1
        elif (inside == 1 and char == '>'):
            inside = 0
        elif inside == 1:
            continue
        else:
            text += char

    return text

That should do it @ElectronGM but let me know if it doesn't. I'm a little surprised you were able to work through the previous lessons in the sequence, because you should have gotten this same error as soon as the stripTags() function was introduced.

@svmelton - let's leave this open for now until I can do a pull request to update the codeblocks and zip folders with the fix? also want to make sure this won't inadvertently introduce new errors later in the sequence, because I've never really looked at these lessons. I'll rename this ticket to reflect the fact that we need a PR for it.

[ElectronGM]

Thanks for the answer. In the past few days, I have spent many hours reviewing the proposed solution and translating it into my hand-made solution. First in the ZIP program: the program now runs. In my solution there is this new error message: Traceback (most recent call last):   File "html-to-pretty-print.py", line 14, in <module>     worddict += obo.nGramsToKWICDict(ngrams) TypeError: 'int' object is not iterable The coding of my program is: # html-to-pretty-print.py import obo # create dictionary of n-grams n = 7 url = 'http://www.oldbaileyonline.org/browse.jsp?id=t17800628-33&div=t17800628-33' text = obo.webPageToText(url) #print('text = ', text) fullwordlist = obo.stripNonAlphaNum(text) #print('fullwordlist = ', fullwordlist) ngrams = obo.getNGrams(fullwordlist, n) #print('ngrams = ', ngrams) worddict = [] print('worddict = ', worddict) worddict += obo.nGramsToKWICDict(ngrams) print('worddict = ', worddict["black"]) In contrast to the coding in the ZIP programs, I have built in many PRINT commands to develop an understanding of the processes. In addition, in order to get to Exercise 9 at all, I had to make these changes to the respective programs: # trial-content.py HTML = response.read().decode('utf-8') # html-to-freq-2.py html = response.read().decode('utf-8') # write-html.py f = open('helloworld.html','w') # write-html-2-windows.py f = open('helloworld.html','w') Unfortunately, I am now far from being able to understand the effects of my auxiliary structures and the new error message. So I stay very sad and disappointed with the tutorial. This is a shame not only for the entire time that I have spent in vain, but also because the basic structure of the tutorial seemed very profitable and logically understandable. But if the basic software Python to version 3 has changed so fundamentally that the tutorial specifications can no longer be used, then a reference to it would have cost me much less time with useless activities. I would therefore prefer if I could get an addition, which coding line in which program how to change or insert, possibly also with information about the respective error messages, if one would need it. Yours ElectronGM Am 18.05.2020 um 15:48 schrieb Brandon Walsh:

…

Thanks for sharing @ElectronGM <https://github.com/ElectronGM> - As @svmelton <https://github.com/svmelton> suggested, this lesson was recently updated to Python. It looks like the process was tested thoroughly for that one section you're working with. The code expects that you'll be working with a string, but the requests .read() function returns a bytes object I believe. You should be able to get it working by adding a single line to the stripTags function to change pageContents to a string like the code wants. Modify the striptags file |def stripTags(pageContents): pageContents = str(pageContents) # this is the new line! startLoc = pageContents.find("<p>") endLoc = pageContents.rfind("<br/>") pageContents = pageContents[startLoc:endLoc] inside = 0 text = '' for char in pageContents: if char == '<': inside = 1 elif (inside == 1 and char == '>'): inside = 0 elif inside == 1: continue else: text += char return text | That should do it @ElectronGM <https://github.com/ElectronGM> but let me know if it doesn't. I'm a little surprised you were able to work through the previous lessons in the sequence, because you should have gotten this same error as soon as the stripTags() function was introduced. @svmelton <https://github.com/svmelton> - let's leave this open for now until I can do a pull request to update the codeblocks and zip folders with the fix? also want to make sure this won't inadvertently introduce new errors later in the sequence, because I've never really looked at these lessons. I'll rename this ticket to reflect the fact that we need a PR for it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1768 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMRJ5WVR47SIXAIF57BEWJ3RSE4B5ANCNFSM4M5IBOLA>.

[acrymble]

@ElectronGM I appreciate your frustration. Learning a new way of thinking and communicating is at times a frustrating experience. It certainly was for me.

You seem to be having trouble isolating the cause of your error messages. If you are hoping to continue building your python skills, it would be worth spending some time on StackOverflow (https://stackoverflow.com/) searching for tips on how to overcome your errors.

This is what we would have to do to help you, so building those skills yourself is a good investment in self learning.

[walshbr]

I might wait to try to work through the remaining lessons until we're able to adequately patch all the things ourselves. I copied the code you shared into my own version of the lessons that I put together as I was assembling this patch, and everything ran fine for me. I think @acrymble is right in this case, and I would double check the code in obo.py in particular, since that was the piece I didn't have access to.

Thanks for reaching out - we've also identified the other errors specific to Windows machines that you point out, and we're in the process of patching them. Thanks again for the feedback - this has been very helpful.

[walshbr]

Just a note that I've renamed this ticket to reflect the particular bug that was initially reported. It overlaps with #1777