Update "Sending e-mail" docs to work with Python 3

[cjerdonek]

The documentation for sending e-mails doesn't seem to work when using Python 3. Also, I can't seem to get e-mail sending working with Python 3 when sending binary (PDF) attachments. The code I'm using worked with Python 2. Details are below.

The Gmail REST API documentation for sending e-mail says to do the following:

raw = base64.urlsafe_b64encode(mime.as_string())
body = {'raw': raw}
messages = service.users().messages()
message = messages.send(userId='me', body=body).execute()

However, this code results in the following error:

File "/Users/chris/dev/.virtualenvs/my_package/lib/python3.4/base64.py", line 121, in urlsafe_b64encode
  return b64encode(s).translate(_urlsafe_encode_translation)
File "/Users/chris/dev/.virtualenvs/my_package/lib/python3.4/base64.py", line 62, in b64encode
  encoded = binascii.b2a_base64(s)[:-1]
TypeError: 'str' does not support the buffer interface

Changing as_string() in the above to as_bytes():

raw = base64.urlsafe_b64encode(mime.as_bytes())
body = {'raw': raw}
messages = service.users().messages()
message = messages.send(userId='me', body=body).execute()

results in the following error:

File "/Users/chris/dev/.virtualenvs/my_package/lib/python3.4/site-packages/googleapiclient/discovery.py", line 691, in method
  actual_path_params, actual_query_params, body_value)
File "/Users/chris/dev/.virtualenvs/my_package/lib/python3.4/site-packages/googleapiclient/model.py", line 149, in request
  body_value = self.serialize(body_value)
File "/Users/chris/dev/.virtualenvs/my_package/lib/python3.4/site-packages/googleapiclient/model.py", line 258, in serialize
  return json.dumps(body_value)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/__init__.py", line 230, in dumps
  return _default_encoder.encode(obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/encoder.py", line 192, in encode
  chunks = self.iterencode(o, _one_shot=True)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/encoder.py", line 250, in iterencode
  return _iterencode(o, 0)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/json/encoder.py", line 173, in default
  raise TypeError(repr(o) + " is not JSON serializable")
TypeError: b'Q29ud...TY9PS0tCg==' is not JSON serializable

If I add raw = raw.decode() to ensure that raw is a string in Python 3 as opposed to bytes, then the e-mail does get sent. However, the PDF attachment now seems to be corrupted:

raw = base64.urlsafe_b64encode(mime.as_bytes())
raw = raw.decode()
body = {'raw': raw}
messages = service.users().messages()
message = messages.send(userId='me', body=body).execute()

[cjerdonek]

Okay, I got it working. When adding attachments, I also needed to add the second line as below:

msg.set_payload(contents)
# Encode the payload using Base64.  This line is from here:
# https://docs.python.org/3/library/email-examples.html
encoders.encode_base64(msg)

This also needs to be updated in the docs for Python (see here). I'm not sure how the docs should handle documenting both Python 2 and Python 3.

[tmatsuo]

FYI I encountered similar issue and my quick hack is:
https://github.com/GoogleCloudPlatform/cloud-pubsub-logging-python/blob/master/pubsub_logging/utils.py#L38

[vinayak-mehta]

Thanks @cjerdonek!

[low-on-mana]

thankyou @cjerdonek for your research !

[romunov]

Saved my hind quarters, thank you. I hope they take this up a notch (soon?).

[joseph-whiting]

Awesome fix @cjerdonek, thanks a lot

[fazerlicourice7]

@cjerdonek You, are A SAVIOR!!!! Thank you so much!

[mhoffa]

@cjerdonek Thank you so much!

[divinedragon]

Just in case still faces an issue like me, here is my code.

with open(file, 'rb') as pdf_file:
      pdf = MIMEBase('application', "pdf")
      pdf.set_payload(pdf_file.read())
      encoders.encode_base64(pdf)
      pdf.add_header('Content-Disposition', 'attachment', filename=os.path.basename(file))

message.attach(pdf)

return {'raw': base64.urlsafe_b64encode(message.as_bytes()).decode()}

[zamai]

Wow, 2 years passed since Apr 20, 2015. But I'm still running into the same issue.
Maybe it's time to update docs?
Thanks, @cjerdonek for your initial investigation and @divinedragon for the example.

[gmarchi]

Still same issue in 2017! Thanks @cjerdonek

[ghost]

Still same issue in October 2017!

here is a full working python 3.6 script for sending messages with attatchments (you just need to fill out the arguments for create_message_with_attatchment):

https://pastebin.com/aCQaVavQ

I spent 2 hours to make it work, quite sad that they don't update their documentation.

[mehdirazajaffri]

Google should update its documentation.

[jawatech]

Still same issue in April 2018!!

@LuaBR your script is the first fully working example I see.

sadly this issue has taken me nearly one day...

[StevenGann]

@LuaBR You are my hero. I spent the whole bloody day trying to fix Google's code.

It's a shame Google doesn't care about developers.

[theacodes]

I apologize that this has slipped under the radar so long. I'm really glad that @LuaBR's solution has been able to help those who've found this issue.

Unfortunately, the people who maintain this library do not maintain those samples. The good news is that we're all at the same company, so I've filed a bug internally with them to update their documentation and samples (internal issue number 78191393)- and I'll happily help them do so.

I'm going to close this issue as there isn't anything actionable for this library. I will comment here when the internal issue about the documentation is addressed.

Thanks, everyone.

[ofersadan85]

Just reporting that this issue still exists as of July 2018, spent hours trying to fix something that doesn't really need more than 3 minutes of docs corrections to help the users and developers who rely on you... isn't it ridiculous that the people writing the documentation are not the same people maintaining the library with all it's changes and issues? This is again the same feeling I had developing in "Google Apps script"... it seems that both here and there you guys don't talk to each other and just file "internal bug reports" that no one ever attends to.

THANK YOU @LuaBR for your solution!

[melvinkcx]

Just reporting ,this is still happening as of 21 Oct 2019.

[zen29d]

Still now the same error. I mean, ohh come on google update your docs.
Thanks @cjerdonek

[busunkim96]

Apologies for all the frustration caused by this. I've pinged the internal bug originally filed by Thea. Thank you for your patience.

[fw-aaron]

Still an issue in February, 2020. I see this issue is closed which is fine. Where can we keep up-to-date on the progress of the documentation issue? On the bright side, my client was only billed about 6 hours for this frustration, not a whole day. That's a win, right!?

[PFython]

Coming somewhat late to the party, but here's my solution if it helps others:

https://stackoverflow.com/a/61929807/7971585

[iwangu]

@PFython that helped a ton, thx for your most recent post!!

[karman1102]

Has anyone figured out how to reply to a given thread using thread id?
I tried the code given here https://stackoverflow.com/questions/58336859/how-to-send-an-existing-email-to-a-new-thread-using-gmail-api. Problem - under the sent mail section, it shows that the email has been replied to whereas it comes as a new mail on the recipient's end.

[fw-aaron]

@karman1102 I'm not sure what all needs to be in line for e-mails to group together, but I don't think it has anything to do with the e-mail itself. Gmail is grouping them on the recipient's side. I was always under the impression that as long as the subject was exactly the same (minus "Re: ", "Fwd: ", etc.), Gmail would group them (unless there's no subject). I've had this happen to me when it's not desired (accidentally-matching subjects like "Invoice"). That stack exchange question and answer certainly seem to suggest a "thread id" 🤷‍♂️ .

[karman1102]

@fw-aaron The use of thread id and subject had been mentioned in the official documentation as well. I wonder if anyone has accomplished the 'grouping' together of mails on the recipient's side.

What I wish to accomplish is - replying from Gmail API to a mail such that it appears as a 'replied-to' mail on the recipient's end. That is, grouped together

[ashwinsr]

Same issue in May 2021!

[vinit1]

Yes, I faced the same issue in May 2021!
Thanks @cjerdonek!