Errors and Exceptions

[edhead123]

Firstly, i want to thank you for the work you've done, it's been incredibly easy to integrate my Google Document as the storage for my data logging.

I'm fairly new to Python, and most of my "coding" consists of finding examples online and tailoring them to work with what I'm looking to do.

My code is running on a Raspberry Pi and designed to pull the indoor/outdoor temperature, and log it to a spreadsheet. When I use URLlib to pull that data, I use the try:, except syntax to catch errors and not crash the program.

Previously, I was updating one cell at a time, which took much longer, and would lock up the program every few days. When I went to my spreadsheet, the row of data would be partially updated. I understood that as the communication between the program and Google failing, causing the program to lock up. I read some of the posts and converted my program to update an entire row at one time, with one API call. That sped up the program, and allowed it to run for longer periods of time before locking up.

That all being said, how would I add a error handler to catch when it fails and prevent it from locking up? Would I do that when I do the login call, the spreadsheet open call, or the update_cells call?

Thank you for any help!

Edward

[burnash]

Hi Edward, thank you for your support. I'm glad gspread helps you.

I haven't experienced any locks in communication while running gspread (it might be not enought running cycles?). Could you please explain in more detail how the lock up in your case looks like? Also can you share a sample or maybe pseudo code from your real code here or email me to fuss.here@gmail.com? That will also help a lot.

[edhead123]

Originally, I was using this code (stripped down a bit for legibility):

#Log Data to Google spreadsheet
gc = gspread.login('email@gmail.com','xxxxxxxxxxxxx')
spreadsheet = gc.open_by_key("xxxxxxxxxxxxxxxx")
worksheet_data = spreadsheet.worksheet("Data")
ColA = worksheet_data.col_values(1)
ColA = [int(x) for x in ColA]
Row = str(max(ColA)+2)

            worksheet_data.update_acell('A'+ Row, str(int(Row)-1))
            worksheet_data.update_acell('B'+ Row, localTime)
            worksheet_data.update_acell('C'+ Row, dateTime)
            worksheet_data.update_acell('D'+ Row, location)
            worksheet_data.update_acell('E'+ Row, outdoorTemp)
            worksheet_data.update_acell('F'+ Row, outdoorHumidity)
            worksheet_data.update_acell('G'+ Row, indoorTemp)
            worksheet_data.update_acell('H'+ Row, fstate)
            worksheet_data.update_acell('I'+ Row, hold)
            worksheet_data.update_acell('J'+ Row, externalIP)

When I was using that code, I would occasionally get some incomplete columns, and the computer freezing up (unable to use locally or ssh). My spreadsheet looked like this:

I would unplug and restart my Raspberry Pi and it would continue working until it happened again. It was random as to which cell it would freeze on, not always the same one.

I was reading one of the other threads and used the update_cells function to speed it up, looking like this:

            cellList = worksheet_data.range('A' + str(Row) + ':J' + str(Row))
            cellList[0].value = str(int(Row)-1)
            cellList[1].value = localTime
            cellList[2].value = dateTime
            cellList[3].value = location
            cellList[4].value = outdoorTemp
            cellList[5].value = outdoorHumidity
            cellList[6].value = indoorTemp
            cellList[7].value = fstate
            cellList[8].value = hold
            cellList[9].value = externalIP
            worksheet_data.update_cells(cellList)

This was much more stable, when it was running for a few days at a time before, I could get this code to run up to a week before locking up. When this code locked up, the entire row didn't show up and was frozen (as I would expect using the update_cells function since it does it all at once).

Thanks for any suggestions!

[ghost]

In the above screenshot of your spreadsheet, did the program have to be restarted in order to continue logging the data? Or did it lock up, and then continue again at the next interval? The spreadsheet makes it look as if you had to manually reset the Raspberry Pi before it continued logging the data, but I just want to verify.

[edhead123]

When it locks up, I have to hard reset the RPi for it to start logging again. I am unable to VNC or SSH in.

Thank for the interest!

[ghost]

There's really no way to 100% know where the program is locking up, but I can probably offer a pretty reasonable guess based on how your program actually works. Are you doing the login call every time you update the temperature, or are you only doing it at the beginning of the program?

[edhead123]

Yes, I do this block of code every time (about every 15 minutes):

gc = gspread.login('email@gmail.com','xxxxxxxxxxxxx')
spreadsheet = gc.open_by_key("xxxxxxxxxxxxxxxx")
worksheet_data = spreadsheet.worksheet("Data")

My thought process was that long term, if I only logged in once when the program is first run, it might log me out at some time, so it would be safer to login every time I update the data.

You think it might have something to do with the logging in?

Thanks!

[ghost]

My initial thought was that maybe you were only logging in once, and that was why the program was terminating. If I were you, I'd throw in some print statements in a couple places and try to find out exactly where the issue is occurring that way, Programming 101 style.

I should also mention that it's very possible that this is an issue on RPi's side, and might not necessarily be due to anything in gspread.

[edhead123]

I definitely agree, it may definitely be something to do with the RPi. I actually had print statements after every step, after login, after opening the spreadsheet and after opening the worksheet. When I had those statements in, it for some reason didn't run as stable. I commented them out and it's been running for about 6 days with no errors so far.

Might be coincidence, but it's doing better than when I had those print statements in. I've been running it headless, but just recently attached a monitor to see. I've been waiting for it to crash again so I can see if there are any messages, but hasn't crashed yet.

I'll definitely report back once I get more info.

Thanks again.

[ghost]

No problem, glad to help!

[burnash]

Hi Edward, thanks for the details. I'd like to qualify Intrepid90's question. What method of running the script are you using? Is it cron-based, Twisted or you have plain ol' while with a time.sleep?

[edhead123]

It's a plain ol while loop with a time.sleep delay at the moment.

[burnash]

From the screenshot it looks like it is about 15 minutes, right?

[edhead123]

Yep! 15 minutes exactly.

[burnash]

Good. My first suggestion would be to try to use cron for scheduling. If you haven't used it before, this is a sample workflow:

1. Run a crontab editor:

   $ crontab -e

1. Replace a dumb path in the following line with your script's filename and paste it to crontab editor:

   */15 * * * * python /path/to/your/script.py >> /dev/null 2>&1

A */15 * * * * part instructs cron to run python /path/to/your/script.py every 15 minutes, while >> /dev/null 2>&1 will discard all script's output (prints and errors). You can replace the /dev/null with some log file name to see what's going on if you need it.

Note: Don't forget to remove while loop and time.sleep from your code.

[edhead123]

Thanks for the suggestion! I took out the loop and it is running as a cron job now. It's been very solid as a while loop for almost a week now, and I hope that it continues as a cron. I will update if it crashes again.

Also, thank you again gspread! I realize that we may have veered away from gspread issues, but I do appreciate the help with my program as a whole (and running linux for the first time).

[burnash]

You're welcome