Skip to content

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brettcannon opened this issue Sep 5, 2015 · 18 comments
Closed

Deprecate smtpd (based on deprecated asyncore/asynchat) #69196

brettcannon opened this issue Sep 5, 2015 · 18 comments
Assignees
Labels
3.7 (EOL) end of life docs Documentation in the Doc dir

Comments

@brettcannon
Copy link
Member

BPO 25008
Nosy @warsaw, @brettcannon, @rhettinger, @terryjreedy, @vstinner, @ericvsmith, @giampaolo, @ned-deily, @bitdancer
PRs
  • bpo-25008: Deprecate smtpd and point to aiosmtpd #274
  • bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) #278
  • Cherry pick bpo-25008 to 3.5 #279
  • Cherry pick bpo-25008 to 3.4 #280
  • [Do Not Merge] Sample of CPython life with blurb. #703
  • [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
  • Files
  • aiosmtpd.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/warsaw'
    closed_at = <Date 2017-02-24.20:32:41.162>
    created_at = <Date 2015-09-05.17:08:13.926>
    labels = ['3.7', 'docs']
    title = 'Deprecate smtpd (based on deprecated asyncore/asynchat)'
    updated_at = <Date 2019-05-10.18:11:02.809>
    user = 'https://github.com/brettcannon'

    bugs.python.org fields:

    activity = <Date 2019-05-10.18:11:02.809>
    actor = 'ned.deily'
    assignee = 'barry'
    closed = True
    closed_date = <Date 2017-02-24.20:32:41.162>
    closer = 'barry'
    components = ['Documentation']
    creation = <Date 2015-09-05.17:08:13.926>
    creator = 'brett.cannon'
    dependencies = []
    files = ['40605']
    hgrepos = []
    issue_num = 25008
    keywords = ['patch']
    message_count = 18.0
    messages = ['249911', '249913', '249935', '249942', '249953', '250019', '250110', '250111', '250149', '250510', '251736', '252285', '253097', '288391', '290408', '290409', '290410', '290411']
    nosy_count = 10.0
    nosy_names = ['barry', 'brett.cannon', 'rhettinger', 'terry.reedy', 'vstinner', 'eric.smith', 'giampaolo.rodola', 'ned.deily', 'r.david.murray', 'neic']
    pr_nums = ['274', '278', '279', '280', '703', '552']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue25008'
    versions = ['Python 3.7']

    @brettcannon
    Copy link
    Member Author

    As mentioned in passing in issue bpo-25002, perhaps smtpd should be deprecated or at least be updated to use asyncio.

    @brettcannon brettcannon added the stdlib Python modules in the Lib dir label Sep 5, 2015
    @brettcannon
    Copy link
    Member Author

    I should mention one of the motivations behind deprecating smtpd is that no one should be using that module to run an SMTP server and it doesn't provide utility like http.server does and so we probably should drop the code to prevent anyone accidentally using it and not have to maintain it.

    @rhettinger
    Copy link
    Contributor

    Bringing in Eric V Smith's comment in
    http://bugs.python.org/issue25002#msg249873 :

    '''
    I use smtpd.py for testing my code. But it's not such a big deal that I couldn't live without it. If I have some time to burn, I might convert it to asyncio so I can continue to use it.

    '''

    Bringing in R David Murray's comment in http://bugs.python.org/issue25002#msg249882 :

    '''smtpd is used for testing smtplib. It is also used in test_logging. I would object to having it removed, but I suspect we can manage to rewrite it by 3.8. Maybe Eric and I can collaborate on it.
    '''

    @rhettinger
    Copy link
    Contributor

    FWIW, I would like to see the module live on (after conversion to asyncio). Doug Hellman's example at https://pymotw.com/2/smtpd/ makes for a nice demonstration of Python's capabilities.

    @warsaw
    Copy link
    Member

    warsaw commented Sep 6, 2015

    Removing smtpd would definitely be a hardship for me right now, probably for obvious reasons. I use it in testing frameworks, and even wrote a library called lazr.smtptest that is built around smtpd. In Mailman, we have an LMTP server built on smtpd that implements *the* official way of getting mail into the system from an upstream mail server. Mailman 3 core FWIW is Python 3.4 and 3.5 compatible.

    That all being said, I've also been thinking lately about an asyncio-based reimplementation of SMTP and LMTP. asyncore/chat and smtpd are not the easiest modules to work with, extend, understand, or debug. I'd be very happy to collaborate with folks on an asyncio-based version. Maybe we can put that out as a third party module, with a long term plan on pulling it into the stdlib and promoting it instead of smtpd.

    @bitdancer
    Copy link
    Member

    I believe that smtpd is used for a *lot* of testing code (anyone who is writing smtp clients in python, pretty much, and probably some writing in other languages). I really don't see the rationale for removing it, assuming we can rewrite it "in time".

    On the other hand, are we really going to *remove* asyncore/asynchat? Or is this a "deprecation in place" like optparse? If the latter then I don't really care if the same is done to smtpd :) If the former, why, if we've kept optparse?

    @vstinner
    Copy link
    Member

    vstinner commented Sep 7, 2015

    Even if it may be possible to support asyncore/asynchat and asyncio at the same time, I would prefer to drop asyncore/asynchat support to simplify the code.

    The problem is that the API will be very different.

    asyncio has a different design. You don't instanciate a class to open a network connection, but you have to call "yield from loop.create_connection()" in a coroutine.

    We might provide helper blocking functions starting the server for us and then running the event loop until CTRL+c is pressed (or another event to stop to stop the loop).

    Rewriting smtpd with asyncio would allow to use it in an asyncio application. In 2015, I expect to see more and more applications using asyncio than asyncore/asynchat.

    Can we modify the issue title to "Rewrite smtpd with asyncio"?

    @warsaw
    Copy link
    Member

    warsaw commented Sep 7, 2015

    On Sep 07, 2015, at 02:26 PM, STINNER Victor wrote:

    Can we modify the issue title to "Rewrite smtpd with asyncio"?

    Sure, although I'm currently thinking it's best to go third party until the
    API and implementation settle down.

    @rhettinger
    Copy link
    Contributor

    Unless there is a drop-in replacement, I don't think this should be deprecated or removed. Our biggest problem in the Python world is getting organizations using Python 2 to switch to Python 3. Deprecating modules that have been a part of the firmament for a long time works against that goal.

    Side note: a quick search shows that people are still writing SMTPD modules in other languages (i.e. Go now has one)

    @terryjreedy
    Copy link
    Member

    http://nullege.com/codes/search?cq=import+smtpd
    shows 135 hits. FWIW, import+asyncore get 972.

    @vstinner vstinner changed the title Deprecate smtpd Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio Sep 18, 2015
    @bitdancer
    Copy link
    Member

    Here is a proof of concept port of smtpd to asyncio. Obviously I'm not suggesting we commit this patch (it isn't complete...the tests don't run). Instead I'm posting it as an outgrowth the of the sprint that was conducted today. The DC team took a different tack, so I'm not submitting this as a patch to their repo, and since it is useful to look at the diff against default, it seems appropriate to post it here.

    The advantage of this port is that it uses the existing smtpd protocol code, which is the "field tested" code. Of course, I did have to change how the data is turned from bytes into lines, so I may have introduced some bugs.

    As discussed, *using* an asyncio based smtpd has a different calling API than using the asynchat one. However, the *pattern* is similar, so it may not be crazy to replace smtpd with the cleaned up version of this patch, exactly because the number of places this is used is relatively small and mostly in test code. IMO it's a better option than deleting smtpd from the stdlib.

    The code as is can be run using its CLI. I tested it via telnet using the DebuggingServer and hand coding an email...for some reason I can't currently import smtplib (Enum errors in socket), so I didn't test it with smtplib, but it should work.

    @warsaw
    Copy link
    Member

    warsaw commented Oct 4, 2015

    I'm liking RDM's port of smtpd.py. I've done some clean ups, bug fixes, and added some debugging. See this branch over on gitlab:

    https://gitlab.com/python-smtpd-hackers/aiosmtpd/tree/issue25508

    Try python3 server.py then run client.py in another terminal (it uses smtplib) or just telnet.

    I plan to continue pursuing this approach; please comment over there.

    @warsaw
    Copy link
    Member

    warsaw commented Oct 16, 2015

    This has come along pretty well. The project is managed here:

    https://gitlab.com/python-smtpd-hackers/aiosmtpd

    and documentation here:

    http://aiosmtpd.readthedocs.org/en/latest/

    @warsaw
    Copy link
    Member

    warsaw commented Feb 22, 2017

    aiosmtpd is coming along nicely:

    http://aiosmtpd.readthedocs.io/en/latest/

    We'll soon have a 1.0 release. Still, I don't think it's worth pulling this into the stdlib. But we could silently deprecate it and point to aiosmtpd in the docs.

    @warsaw warsaw added docs Documentation in the Doc dir 3.7 (EOL) end of life and removed stdlib Python modules in the Lib dir labels Feb 22, 2017
    @warsaw warsaw changed the title Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio Deprecate smtpd (based on deprecated asyncore/asynchat) Feb 22, 2017
    @warsaw warsaw self-assigned this Feb 22, 2017
    @warsaw warsaw closed this as completed Feb 24, 2017
    @warsaw
    Copy link
    Member

    warsaw commented Mar 24, 2017

    New changeset 66b5092 by Barry Warsaw in branch '3.5':
    bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#279)
    66b5092

    @warsaw
    Copy link
    Member

    warsaw commented Mar 24, 2017

    New changeset 0a1b656 by Barry Warsaw in branch '3.6':
    bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#278)
    0a1b656

    @warsaw
    Copy link
    Member

    warsaw commented Mar 24, 2017

    New changeset f37b0cb by Barry Warsaw in branch '3.4':
    bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#280)
    f37b0cb

    @vstinner
    Copy link
    Member

    New changeset fba79dc by Victor Stinner (Barry Warsaw) in branch 'master':
    bpo-25008: Deprecate smtpd and point to aiosmtpd (#274)
    fba79dc

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life docs Documentation in the Doc dir
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants