Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

brettcannon · 2015-09-05T17:08:14Z

BPO	25008
Nosy	@warsaw, @brettcannon, @rhettinger, @terryjreedy, @vstinner, @ericvsmith, @giampaolo, @ned-deily, @bitdancer
PRs	bpo-25008: Deprecate smtpd and point to aiosmtpd #274 bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) #278 Cherry pick bpo-25008 to 3.5 #279 Cherry pick bpo-25008 to 3.4 #280 [Do Not Merge] Sample of CPython life with blurb. #703 [Do Not Merge] Convert `Misc/NEWS` so that it is managed by towncrier #552
Files	aiosmtpd.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/warsaw'
closed_at = <Date 2017-02-24.20:32:41.162>
created_at = <Date 2015-09-05.17:08:13.926>
labels = ['3.7', 'docs']
title = 'Deprecate smtpd (based on deprecated asyncore/asynchat)'
updated_at = <Date 2019-05-10.18:11:02.809>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2019-05-10.18:11:02.809>
actor = 'ned.deily'
assignee = 'barry'
closed = True
closed_date = <Date 2017-02-24.20:32:41.162>
closer = 'barry'
components = ['Documentation']
creation = <Date 2015-09-05.17:08:13.926>
creator = 'brett.cannon'
dependencies = []
files = ['40605']
hgrepos = []
issue_num = 25008
keywords = ['patch']
message_count = 18.0
messages = ['249911', '249913', '249935', '249942', '249953', '250019', '250110', '250111', '250149', '250510', '251736', '252285', '253097', '288391', '290408', '290409', '290410', '290411']
nosy_count = 10.0
nosy_names = ['barry', 'brett.cannon', 'rhettinger', 'terry.reedy', 'vstinner', 'eric.smith', 'giampaolo.rodola', 'ned.deily', 'r.david.murray', 'neic']
pr_nums = ['274', '278', '279', '280', '703', '552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue25008'
versions = ['Python 3.7']

The text was updated successfully, but these errors were encountered:

brettcannon · 2015-09-05T17:08:14Z

As mentioned in passing in issue bpo-25002, perhaps smtpd should be deprecated or at least be updated to use asyncio.

brettcannon · 2015-09-05T17:10:08Z

I should mention one of the motivations behind deprecating smtpd is that no one should be using that module to run an SMTP server and it doesn't provide utility like http.server does and so we probably should drop the code to prevent anyone accidentally using it and not have to maintain it.

rhettinger · 2015-09-05T22:05:26Z

Bringing in Eric V Smith's comment in
http://bugs.python.org/issue25002#msg249873 :

'''
I use smtpd.py for testing my code. But it's not such a big deal that I couldn't live without it. If I have some time to burn, I might convert it to asyncio so I can continue to use it.

'''

Bringing in R David Murray's comment in http://bugs.python.org/issue25002#msg249882 :

'''smtpd is used for testing smtplib. It is also used in test_logging. I would object to having it removed, but I suspect we can manage to rewrite it by 3.8. Maybe Eric and I can collaborate on it.
'''

rhettinger · 2015-09-05T22:32:56Z

FWIW, I would like to see the module live on (after conversion to asyncio). Doug Hellman's example at https://pymotw.com/2/smtpd/ makes for a nice demonstration of Python's capabilities.

warsaw · 2015-09-06T02:51:48Z

Removing smtpd would definitely be a hardship for me right now, probably for obvious reasons. I use it in testing frameworks, and even wrote a library called lazr.smtptest that is built around smtpd. In Mailman, we have an LMTP server built on smtpd that implements *the* official way of getting mail into the system from an upstream mail server. Mailman 3 core FWIW is Python 3.4 and 3.5 compatible.

That all being said, I've also been thinking lately about an asyncio-based reimplementation of SMTP and LMTP. asyncore/chat and smtpd are not the easiest modules to work with, extend, understand, or debug. I'd be very happy to collaborate with folks on an asyncio-based version. Maybe we can put that out as a third party module, with a long term plan on pulling it into the stdlib and promoting it instead of smtpd.

bitdancer · 2015-09-06T20:38:15Z

I believe that smtpd is used for a *lot* of testing code (anyone who is writing smtp clients in python, pretty much, and probably some writing in other languages). I really don't see the rationale for removing it, assuming we can rewrite it "in time".

On the other hand, are we really going to *remove* asyncore/asynchat? Or is this a "deprecation in place" like optparse? If the latter then I don't really care if the same is done to smtpd :) If the former, why, if we've kept optparse?

vstinner · 2015-09-07T14:26:49Z

Even if it may be possible to support asyncore/asynchat and asyncio at the same time, I would prefer to drop asyncore/asynchat support to simplify the code.

The problem is that the API will be very different.

asyncio has a different design. You don't instanciate a class to open a network connection, but you have to call "yield from loop.create_connection()" in a coroutine.

We might provide helper blocking functions starting the server for us and then running the event loop until CTRL+c is pressed (or another event to stop to stop the loop).

Rewriting smtpd with asyncio would allow to use it in an asyncio application. In 2015, I expect to see more and more applications using asyncio than asyncore/asynchat.

Can we modify the issue title to "Rewrite smtpd with asyncio"?

warsaw · 2015-09-07T14:34:13Z

On Sep 07, 2015, at 02:26 PM, STINNER Victor wrote:

Can we modify the issue title to "Rewrite smtpd with asyncio"?

Sure, although I'm currently thinking it's best to go third party until the
API and implementation settle down.

rhettinger · 2015-09-08T03:25:02Z

Unless there is a drop-in replacement, I don't think this should be deprecated or removed. Our biggest problem in the Python world is getting organizations using Python 2 to switch to Python 3. Deprecating modules that have been a part of the firmament for a long time works against that goal.

Side note: a quick search shows that people are still writing SMTPD modules in other languages (i.e. Go now has one)

terryjreedy · 2015-09-12T03:22:56Z

http://nullege.com/codes/search?cq=import+smtpd
shows 135 hits. FWIW, import+asyncore get 972.

bitdancer · 2015-09-28T00:28:08Z

Here is a proof of concept port of smtpd to asyncio. Obviously I'm not suggesting we commit this patch (it isn't complete...the tests don't run). Instead I'm posting it as an outgrowth the of the sprint that was conducted today. The DC team took a different tack, so I'm not submitting this as a patch to their repo, and since it is useful to look at the diff against default, it seems appropriate to post it here.

The advantage of this port is that it uses the existing smtpd protocol code, which is the "field tested" code. Of course, I did have to change how the data is turned from bytes into lines, so I may have introduced some bugs.

As discussed, *using* an asyncio based smtpd has a different calling API than using the asynchat one. However, the *pattern* is similar, so it may not be crazy to replace smtpd with the cleaned up version of this patch, exactly because the number of places this is used is relatively small and mostly in test code. IMO it's a better option than deleting smtpd from the stdlib.

The code as is can be run using its CLI. I tested it via telnet using the DebuggingServer and hand coding an email...for some reason I can't currently import smtplib (Enum errors in socket), so I didn't test it with smtplib, but it should work.

warsaw · 2015-10-04T20:15:17Z

I'm liking RDM's port of smtpd.py. I've done some clean ups, bug fixes, and added some debugging. See this branch over on gitlab:

https://gitlab.com/python-smtpd-hackers/aiosmtpd/tree/issue25508

Try python3 server.py then run client.py in another terminal (it uses smtplib) or just telnet.

I plan to continue pursuing this approach; please comment over there.

warsaw · 2015-10-16T20:51:21Z

This has come along pretty well. The project is managed here:

https://gitlab.com/python-smtpd-hackers/aiosmtpd

and documentation here:

http://aiosmtpd.readthedocs.org/en/latest/

warsaw · 2017-02-22T23:10:45Z

aiosmtpd is coming along nicely:

http://aiosmtpd.readthedocs.io/en/latest/

We'll soon have a 1.0 release. Still, I don't think it's worth pulling this into the stdlib. But we could silently deprecate it and point to aiosmtpd in the docs.

warsaw · 2017-03-24T23:47:47Z

New changeset 66b5092 by Barry Warsaw in branch '3.5':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#279)
66b5092

warsaw · 2017-03-24T23:47:54Z

New changeset 0a1b656 by Barry Warsaw in branch '3.6':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#278)
0a1b656

warsaw · 2017-03-24T23:48:02Z

New changeset f37b0cb by Barry Warsaw in branch '3.4':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#280)
f37b0cb

vstinner · 2017-03-24T23:48:17Z

New changeset fba79dc by Victor Stinner (Barry Warsaw) in branch 'master':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274)
fba79dc

brettcannon added the stdlib Python modules in the Lib dir label Sep 5, 2015

vstinner changed the title ~~Deprecate smtpd~~ Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio Sep 18, 2015

warsaw added docs Documentation in the Doc dir 3.7 (EOL) end of life and removed stdlib Python modules in the Lib dir labels Feb 22, 2017

warsaw changed the title ~~Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio~~ Deprecate smtpd (based on deprecated asyncore/asynchat) Feb 22, 2017

warsaw self-assigned this Feb 22, 2017

warsaw closed this as completed Feb 24, 2017

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

brettcannon commented Sep 5, 2015

brettcannon commented Sep 5, 2015

brettcannon commented Sep 5, 2015

rhettinger commented Sep 5, 2015

rhettinger commented Sep 5, 2015

warsaw commented Sep 6, 2015

bitdancer commented Sep 6, 2015

vstinner commented Sep 7, 2015

warsaw commented Sep 7, 2015

rhettinger commented Sep 8, 2015

terryjreedy commented Sep 12, 2015

bitdancer commented Sep 28, 2015

warsaw commented Oct 4, 2015

warsaw commented Oct 16, 2015

warsaw commented Feb 22, 2017

warsaw commented Mar 24, 2017

warsaw commented Mar 24, 2017

warsaw commented Mar 24, 2017

vstinner commented Mar 24, 2017

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

Comments

brettcannon commented Sep 5, 2015

brettcannon commented Sep 5, 2015

brettcannon commented Sep 5, 2015

rhettinger commented Sep 5, 2015

rhettinger commented Sep 5, 2015

warsaw commented Sep 6, 2015

bitdancer commented Sep 6, 2015

vstinner commented Sep 7, 2015

warsaw commented Sep 7, 2015

rhettinger commented Sep 8, 2015

terryjreedy commented Sep 12, 2015

bitdancer commented Sep 28, 2015

warsaw commented Oct 4, 2015

warsaw commented Oct 16, 2015

warsaw commented Feb 22, 2017

warsaw commented Mar 24, 2017

warsaw commented Mar 24, 2017

warsaw commented Mar 24, 2017

vstinner commented Mar 24, 2017