The port is properly parsed from the email_url, but then it's not used. AFAICT, there's really no way starttls would work without it. On my local site-packages version, I changed it to be:
if proto == 'ssl':
log.debug("Setting up SSL")
session = SMTP_SSL(self._conf['fqdn'], self._conf['port'])
session = SMTP(self._conf['fqdn'], self._conf['port'])
Sorry I'm too busy to submit a proper patch.
And just an fyi, I tested this and it worked with gmail starttls smtp server.
Enable configurable SMTP port. Closes: #43