Skip to content
This repository

Better bulk sending #19

Merged
merged 6 commits into from about 2 years ago

2 participants

Matt Robenolt David Martorana
Matt Robenolt

This pull request focuses on sending bulk mail properly.

  1. The Django backend properly uses the PMBatchMail class instead of iterating over a list.
  2. PMBatchMail assures that batch email can only be sent 500 at a time. If more emails are attempted to be sent, messages are broken up into chunks of 500 for sending.

There are also a couple minor things:

  1. All requests are done over https instead of http. I don't see why we'd want to use http when https is available for all calls.
  2. Adjusted the Django test project to import the postmark package from within our directory.
  3. Replaced your API key with the proper testing key as documented: http://developer.postmarkapp.com/developer-build.html#authentication-headers
Matt Robenolt mattrobenolt Let's fix up the Django demo so it can import the postmark package, a…
…nd strip out your API key
9981c52
Matt Robenolt mattrobenolt Why use http when https is available? a38a261
Matt Robenolt mattrobenolt Support for proper bulk sending from Django's send_mass_mail function
Constructs an actual PMBatchMail object to send instead of many
individual calls to PMMail.send()
5ce4029
Matt Robenolt mattrobenolt Split batch requests into chunks of 500 messages since 500 is the max…
… that can be sent at once
e2d106a
Matt Robenolt mattrobenolt Added a constant for the maximum number of messages to be sent in a b…
…atch request, and added some comments
c2809f6
Matt Robenolt mattrobenolt Actual filter out messages that don't have recipients
_build_message will return False if no recipients are specified, so we
want to filter those out of the bulk messages and bail before even
attempting to send.
824514b
David Martorana

Hey Matt!

Just wanted to let you know that I'm not ignoring your pull request - I just haven't had time to get around to it. I'll try to incorporate it in the near future.

Thanks!

Dave

Matt Robenolt

No worries. We're using my fork of this in our own project and it's working fine so far.

Matt Robenolt

Any word on this? I'd like to see another version at least pushed to PyPi with the ugettext fix in for Django. ;)

Matt Robenolt

bump

David Martorana themartorana merged commit b68a1b4 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 6 unique commits by 1 author.

Feb 25, 2012
Matt Robenolt mattrobenolt Let's fix up the Django demo so it can import the postmark package, a…
…nd strip out your API key
9981c52
Matt Robenolt mattrobenolt Why use http when https is available? a38a261
Matt Robenolt mattrobenolt Support for proper bulk sending from Django's send_mass_mail function
Constructs an actual PMBatchMail object to send instead of many
individual calls to PMMail.send()
5ce4029
Matt Robenolt mattrobenolt Split batch requests into chunks of 500 messages since 500 is the max…
… that can be sent at once
e2d106a
Matt Robenolt mattrobenolt Added a constant for the maximum number of messages to be sent in a b…
…atch request, and added some comments
c2809f6
Matt Robenolt mattrobenolt Actual filter out messages that don't have recipients
_build_message will return False if no recipients are specified, so we
want to filter those out of the bulk messages and bail before even
attempting to send.
824514b
This page is out of date. Refresh to see the latest.
110 postmark/core.py
@@ -40,7 +40,7 @@ def default(self, o):
40 40
41 41 #
42 42 #
43   -__POSTMARK_URL__ = 'http://api.postmarkapp.com/'
  43 +__POSTMARK_URL__ = 'https://api.postmarkapp.com/'
44 44
45 45 class PMMail(object):
46 46 '''
@@ -435,9 +435,15 @@ def send(self, test=None):
435 435 raise PMMailURLException('URLError: %d: The server couldn\'t fufill the request. (See "inner_exception" for details)' % err.code, err)
436 436 else:
437 437 raise PMMailURLException('URLError: The server couldn\'t fufill the request. (See "inner_exception" for details)', err)
438   -
  438 +
  439 +# Simple utility that returns a generator to chunk up a list into equal parts
  440 +def _chunks(l, n):
  441 + return (l[i:i+n] for i in range(0, len(l), n))
439 442
440 443 class PMBatchMail(object):
  444 + # Maximum number of messages to be sent at once.
  445 + # Ref: http://developer.postmarkapp.com/developer-build.html#batching-messages
  446 + MAX_MESSAGES = 500
441 447
442 448 def __init__(self, **kwargs):
443 449 self.__api_key = None
@@ -469,21 +475,6 @@ def __init__(self, **kwargs):
469 475
470 476
471 477 def send(self, test=None):
472   - json_message = []
473   - for message in self.messages:
474   - json_message.append(message.to_json_message())
475   -
476   - req = urllib2.Request(
477   - __POSTMARK_URL__ + 'email/batch',
478   - json.dumps(json_message, cls=PMJSONEncoder),
479   - {
480   - 'Accept': 'application/json',
481   - 'Content-Type': 'application/json',
482   - 'X-Postmark-Server-Token': self.__api_key,
483   - 'User-agent': self.__user_agent
484   - }
485   - )
486   -
487 478 # If test is not specified, attempt to read the Django setting
488 479 if test is None:
489 480 try:
@@ -491,40 +482,57 @@ def send(self, test=None):
491 482 test = getattr(django_settings, "POSTMARK_TEST_MODE", None)
492 483 except ImportError:
493 484 pass
494   -
495   - # If this is a test, just print the message
496   - if test:
497   - print 'JSON message is:\n%s' % json.dumps(json_message, cls=PMJSONEncoder)
498   - return
499   -
500   - # Attempt send
501   - try:
502   - result = urllib2.urlopen(req)
503   - result.close()
504   - if result.code == 200:
505   - return True
506   - else:
507   - raise PMMailSendException('Return code %d: %s' % (result.code, result.msg))
508   - except urllib2.HTTPError, err:
509   - if err.code == 401:
510   - raise PMMailUnauthorizedException('Sending Unauthorized - incorrect API key.', err)
511   - elif err.code == 422:
512   - try:
513   - jsontxt = err.read()
514   - jsonobj = json.loads(jsontxt)
515   - desc = jsonobj['Message']
516   - except:
517   - desc = 'Description not given'
518   - raise PMMailUnprocessableEntityException('Unprocessable Entity: %s' % desc)
519   - elif err.code == 500:
520   - raise PMMailServerErrorException('Internal server error at Postmark. Admins have been alerted.', err)
521   - except urllib2.URLError, err:
522   - if hasattr(err, 'reason'):
523   - raise PMMailURLException('URLError: Failed to reach the server: %s (See "inner_exception" for details)' % err.reason, err)
524   - elif hasattr(err, 'code'):
525   - raise PMMailURLException('URLError: %d: The server couldn\'t fufill the request. (See "inner_exception" for details)' % err.code, err)
526   - else:
527   - raise PMMailURLException('URLError: The server couldn\'t fufill the request. (See "inner_exception" for details)', err)
  485 + # Split up into groups of 500 messages for sending
  486 + for messages in _chunks(self.messages, PMBatchMail.MAX_MESSAGES):
  487 + json_message = []
  488 + for message in messages:
  489 + json_message.append(message.to_json_message())
  490 +
  491 + req = urllib2.Request(
  492 + __POSTMARK_URL__ + 'email/batch',
  493 + json.dumps(json_message, cls=PMJSONEncoder),
  494 + {
  495 + 'Accept': 'application/json',
  496 + 'Content-Type': 'application/json',
  497 + 'X-Postmark-Server-Token': self.__api_key,
  498 + 'User-agent': self.__user_agent
  499 + }
  500 + )
  501 +
  502 + # If this is a test, just print the message
  503 + if test:
  504 + print 'JSON message is:\n%s' % json.dumps(json_message, cls=PMJSONEncoder)
  505 + continue
  506 +
  507 + # Attempt send
  508 + try:
  509 + result = urllib2.urlopen(req)
  510 + result.close()
  511 + if result.code == 200:
  512 + pass
  513 + else:
  514 + raise PMMailSendException('Return code %d: %s' % (result.code, result.msg))
  515 + except urllib2.HTTPError, err:
  516 + if err.code == 401:
  517 + raise PMMailUnauthorizedException('Sending Unauthorized - incorrect API key.', err)
  518 + elif err.code == 422:
  519 + try:
  520 + jsontxt = err.read()
  521 + jsonobj = json.loads(jsontxt)
  522 + desc = jsonobj['Message']
  523 + except:
  524 + desc = 'Description not given'
  525 + raise PMMailUnprocessableEntityException('Unprocessable Entity: %s' % desc)
  526 + elif err.code == 500:
  527 + raise PMMailServerErrorException('Internal server error at Postmark. Admins have been alerted.', err)
  528 + except urllib2.URLError, err:
  529 + if hasattr(err, 'reason'):
  530 + raise PMMailURLException('URLError: Failed to reach the server: %s (See "inner_exception" for details)' % err.reason, err)
  531 + elif hasattr(err, 'code'):
  532 + raise PMMailURLException('URLError: %d: The server couldn\'t fufill the request. (See "inner_exception" for details)' % err.code, err)
  533 + else:
  534 + raise PMMailURLException('URLError: The server couldn\'t fufill the request. (See "inner_exception" for details)', err)
  535 + return True
528 536
529 537
530 538
106 postmark/django_backend.py
@@ -3,7 +3,7 @@
3 3 from django.core.exceptions import ImproperlyConfigured
4 4 from django.core.mail import EmailMessage, EmailMultiAlternatives
5 5
6   -from core import PMMail
  6 +from core import PMMail, PMBatchMail
7 7
8 8 class PMEmailMessage(EmailMessage):
9 9 def __init__(self, *args, **kwargs):
@@ -45,58 +45,72 @@ def send_messages(self, email_messages):
45 45 """
46 46 if not email_messages:
47 47 return
48   - num_sent = 0
49   - for message in email_messages:
50   - sent = self._send(message)
51   - if sent:
52   - num_sent += 1
53   - return num_sent
  48 + sent = self._send(email_messages)
  49 + if sent:
  50 + return len(email_messages)
  51 + return 0
54 52
55 53
56   - def _send(self, message):
57   - """A helper method that does the actual sending."""
  54 + def _build_message(self, message):
  55 + """A helper method to convert a PMEmailMessage to a PMMail"""
58 56 if not message.recipients():
59 57 return False
  58 + recipients = ','.join(message.to)
  59 + recipients_bcc = ','.join(message.bcc)
  60 +
  61 + html_body = None
  62 + if isinstance(message, EmailMultiAlternatives):
  63 + for alt in message.alternatives:
  64 + if alt[1] == "text/html":
  65 + html_body=alt[0]
  66 + break
  67 +
  68 + reply_to = None
  69 + custom_headers = {}
  70 + if message.extra_headers and isinstance(message.extra_headers, dict):
  71 + if message.extra_headers.has_key('Reply-To'):
  72 + reply_to = message.extra_headers.pop('Reply-To')
  73 + if len(message.extra_headers):
  74 + custom_headers = message.extra_headers
  75 + attachments = []
  76 + if message.attachments and isinstance(message.attachments, list):
  77 + if len(message.attachments):
  78 + attachments = message.attachments
  79 +
  80 + postmark_message = PMMail(api_key=self.api_key,
  81 + subject=message.subject,
  82 + sender=message.from_email,
  83 + to=recipients,
  84 + bcc=recipients_bcc,
  85 + text_body=message.body,
  86 + html_body=html_body,
  87 + reply_to=reply_to,
  88 + custom_headers=custom_headers,
  89 + attachments=attachments)
  90 +
  91 + postmark_message.tag = getattr(message, 'tag', None)
  92 + return postmark_message
  93 +
  94 + def _send(self, messages):
  95 + """A helper method that does the actual sending."""
  96 + if len(messages) == 1:
  97 + to_send = self._build_message(messages[0])
  98 + if to_send == False:
  99 + # The message was missing recipients.
  100 + # Bail.
  101 + return False
  102 + else:
  103 + pm_messages = map(self._build_message, messages)
  104 + pm_messages = filter(lambda m: m != False, pm_messages)
  105 + if len(pm_messages) == 0:
  106 + # If after filtering, there aren't any messages
  107 + # to send, bail.
  108 + return False
  109 + to_send = PMBatchMail(messages=pm_messages)
60 110 try:
61   - recipients = ','.join(message.to)
62   - recipients_bcc = ','.join(message.bcc)
63   -
64   - html_body = None
65   - if isinstance(message, EmailMultiAlternatives):
66   - for alt in message.alternatives:
67   - if alt[1] == "text/html":
68   - html_body=alt[0]
69   - break
70   -
71   - reply_to = None
72   - custom_headers = {}
73   - if message.extra_headers and isinstance(message.extra_headers, dict):
74   - if message.extra_headers.has_key('Reply-To'):
75   - reply_to = message.extra_headers.pop('Reply-To')
76   - if len(message.extra_headers):
77   - custom_headers = message.extra_headers
78   - attachments = []
79   - if message.attachments and isinstance(message.attachments, list):
80   - if len(message.attachments):
81   - attachments = message.attachments
82   -
83   - postmark_message = PMMail(api_key=self.api_key,
84   - subject=message.subject,
85   - sender=message.from_email,
86   - to=recipients,
87   - bcc=recipients_bcc,
88   - text_body=message.body,
89   - html_body=html_body,
90   - reply_to=reply_to,
91   - custom_headers=custom_headers,
92   - attachments=attachments)
93   -
94   - postmark_message.tag = getattr(message, 'tag', None)
95   -
96   - postmark_message.send(test=self.test_mode)
  111 + to_send.send(test=self.test_mode)
97 112 except:
98 113 if self.fail_silently:
99 114 return False
100 115 raise
101 116 return True
102   -
7 test/demo/settings.py
@@ -2,6 +2,9 @@
2 2
3 3 import os
4 4 settings_path, settings_module = os.path.split(__file__)
  5 +
  6 +import sys
  7 +sys.path.append('../../')
5 8
6 9 DEBUG = True
7 10 #TEMPLATE_DEBUG = DEBUG
@@ -26,5 +29,5 @@
26 29 #EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
27 30 #EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
28 31 EMAIL_BACKEND = 'postmark.django_backend.EmailBackend'
29   -POSTMARK_API_KEY = '1dd7e294-1059-43e8-ac32-d1ce25378223'
30   -POSTMARK_SENDER = 'system@playcompete.com'
  32 +POSTMARK_API_KEY = 'POSTMARK_API_TEST'
  33 +POSTMARK_SENDER = 'system@playcompete.com'

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.