Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow registering at-fork handlers #60704

Closed
tiran opened this issue Nov 18, 2012 · 58 comments
Closed

Allow registering at-fork handlers #60704

tiran opened this issue Nov 18, 2012 · 58 comments
Assignees
Labels
3.7 extension-modules interpreter-core type-feature

Comments

@tiran
Copy link
Member

@tiran tiran commented Nov 18, 2012

BPO 16500
Nosy @malemburg, @Yhg1s, @birkenfeld, @gpshead, @jcea, @amauryfa, @pitrou, @vstinner, @tiran, @asvetlov, @socketpair, @serhiy-storchaka, @1st1, @ajdavis
PRs
  • #1715
  • #1834
  • #1841
  • #1843
  • #3516
  • #3519
  • Files
  • pure-python-atfork.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/gpshead'
    closed_at = <Date 2017-05-29.17:05:38.621>
    created_at = <Date 2012-11-18.15:20:05.282>
    labels = ['extension-modules', 'interpreter-core', 'type-feature', '3.7']
    title = 'Allow registering at-fork handlers'
    updated_at = <Date 2017-09-12.23:14:11.915>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2017-09-12.23:14:11.915>
    actor = 'vstinner'
    assignee = 'gregory.p.smith'
    closed = True
    closed_date = <Date 2017-05-29.17:05:38.621>
    closer = 'gregory.p.smith'
    components = ['Extension Modules', 'Interpreter Core']
    creation = <Date 2012-11-18.15:20:05.282>
    creator = 'christian.heimes'
    dependencies = []
    files = ['28044']
    hgrepos = []
    issue_num = 16500
    keywords = ['patch', 'needs review']
    message_count = 58.0
    messages = ['175878', '175892', '175967', '175972', '175973', '175974', '175975', '175980', '175997', '176002', '176004', '176019', '176020', '176022', '179838', '179888', '179927', '179945', '179949', '200767', '200774', '200777', '200778', '200779', '200780', '200789', '200797', '200826', '200843', '201892', '201899', '201909', '201912', '266590', '294141', '294145', '294146', '294147', '294148', '294149', '294159', '294597', '294629', '294633', '294634', '294635', '294639', '294640', '294641', '294642', '294651', '294702', '294703', '294705', '294711', '294724', '301983', '302006']
    nosy_count = 23.0
    nosy_names = ['lemburg', 'twouters', 'georg.brandl', 'gregory.p.smith', 'jcea', 'amaury.forgeotdarc', 'pitrou', 'vstinner', 'christian.heimes', 'grahamd', 'Arfrever', 'ionelmc', 'asvetlov', 'neologix', 'socketpair', 'sbt', 'aliles', 'serhiy.storchaka', 'yselivanov', 'DLitz', 'emptysquare', 'xupeng', 'rpcope1']
    pr_nums = ['1715', '1834', '1841', '1843', '3516', '3519']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue16500'
    versions = ['Python 3.7']

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 18, 2012

    I propose the addition of an 'afterfork' module. The module shall fulfill a similar task as the 'atexit' module except that it handles process forks instead of process shutdown.

    The 'afterfork' module shall allow libraries to register callbacks that are executed on fork() inside the child process and as soon as possible. Python already has a function that must be called by C code: PyOS_AfterFork(). The 'afterfork' callbacks are called as the last step in PyOS_AfterFork().

    Use case example:
    The tempfile module has a specialized RNG that re-initialized the RNG after fork() by comparing os.getpid() to an instance variable every time the RNG is accessed. The check can be replaced with an afterfork callback.

    Open questions:
    How should the afterfork() module handle exceptions that are raised by callbacks?

    Implementation:
    I'm going to use as much code from atexitmodule.c as possible. I'm going to copy common code to a template file and include the template from atexitmodule.c and afterforkmodule.c with some preprocessor tricks.

    @tiran tiran self-assigned this Nov 18, 2012
    @tiran tiran added extension-modules interpreter-core type-feature labels Nov 18, 2012
    @sbt
    Copy link
    Mannequin

    @sbt sbt mannequin commented Nov 18, 2012

    pthread_atfork() allows the registering of three types of callbacks:

    1. prepare callbacks which are called before the fork,
    2. parent callbacks which are called in the parent after the fork
    3. child callbacks which are called in the child after the fork.

    I think all three should be supported.

    I also think that a recursive "fork lock" should be introduced which is held during the fork. This can be acquired around critical sections during which forks must not occur.

    This is more or less a duplicate of bpo-6923. See also bpo-6721.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 19, 2012

    Thanks Richard!

    My first reaction was YAGNI but after I read the two tickets I now understand the need for three different hooks. I suggest that we implement our own hooks like the http://linux.die.net/man/3/pthread_atfork function, especially the order of function calls:

    The parent and child fork handlers shall be called in the order in which they were established by calls to pthread_atfork(). The prepare fork handlers shall be called in the opposite order.

    I like to focus on three hooks + the Python API and leave the usage of the hooks to other developers.

    Proposal:

    • Introduce a new module called atfork (Modules/atforkmodule.c) that is build into the core.
    • Move PyOS_AfterFork to Modules/atforkmodule.c.
    • Add PyOS_BeforeFork() (or PyOS_PrepareFork() ?) and PyOS_AfterForkParent()
    • call the two new methods around the calls to fork() in the stdlib.

    I'm not yet sure how to implement the Python API. I could either implement six methods:

      atfork.register_before_fork(callable, *args, **kwargs)
      atfork.register_after_fork_child(callable, *args, **kwargs)
      atfork.register_after_fork_parent(callable, *args, **kwargs)
      atfork.unregister_before_fork(callable)
      atfork.unregister_after_fork_child(callable)
      atfork.unregister_after_fork_parent(callable)

    or two:

      atfork.register(prepare=None, parent=None, child=None, *args, **kwargs)
      atfork.unregister(prepare=None, parent=None, child=None)

    @sbt
    Copy link
    Mannequin

    @sbt sbt mannequin commented Nov 19, 2012

    Note that Gregory P. Smith has written

    http://code.google.com/p/python-atfork/
    

    I also started a pure python patch but did not get round it posting it. (It also implements the fork lock idea.) I'll attach it here.

    How do you intend to handle the propagation of exceptions? I decided that after

        atfork.atfork(prepare1, parent1, child1)
        atfork.atfork(prepare2, parent2, child2)
        ...
        atfork.atfork(prepareN, parentN, childN)

    calling "pid = os.fork()" should be equivalent to

        pid = None
        prepareN()
        try:
            ...
                prepare2()
                try:
                    prepare1()
                    try:
                        pid = posix.fork()
                    finally:
                        parent1() if pid != 0 else child1()
                finally:
                    parent2() if pid != 0 else child2()
            ...
        finally:
            parentN() if pid != 0 else childN()

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented Nov 19, 2012

    I would not allow exceptions to propagate. No caller is expecting them.

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented Nov 20, 2012

    pthread_atfork() cannot be used to implement this. Another non-python
    thread started by a C extension module or the C application that is
    embedding Python within it is always free to call fork() on its own with
    zero knowledge that Python even exists at all. It's guaranteed that fork
    will be called while the Python GIL is held in this situation which would
    cause any pre-fork thing registered by Python to deadlock.

    At best, this can be implemented manually as we do with some of the before
    and after fork stuff today but it must come with the caveat warning that it
    cannot guarantee that these things are actually called before and after
    fork() other than direct os.fork() calls from Python code or extremely
    Python aware C extension modules that may call fork() (very rare, most C &
    C++ libraries an extension module may be using assume that they've got the
    run of the house). ie: this problem is unsolvable unless you control 100%
    of the code being used by your entire user application.

    On Mon, Nov 19, 2012 at 3:59 PM, Gregory P. Smith <report@bugs.python.org>wrote:

    Gregory P. Smith added the comment:

    I would not allow exceptions to propagate. No caller is expecting them.

    ----------


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue16500\>


    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 20, 2012

    Meh! Exception handling takes all the fun of the API and is going to make it MUCH more complicated. pthread_atfork() ignores error handling for a good reason. It's going to be hard to get it right. :/

    IFF we are going to walk the hard and rocky road of exception handling, then we are going to need at least four hooks and a register function that takres four callables as arguments: register(prepare, error, parent, child). Each prepare() call pushes an error handling onto a stack. In case of an exception in a prepare handler, the error stack is popped until all error handlers are called. This approach allows a prepare handler to actually prevent a fork() call from succeeding.

    The parent and child hooks are always called no matter what. Exception are recorded and a warning is emitted when at least one hook fails. We might raise an exception but it has to be a special exception that ships information if fork() has succeeded, if the code runs in child or parent and about the child's PID.

    I fear it's going to be *really* hard to get everything right.

    Gregory made a good point, too. We can rely on pthread_atfork() as we are unable to predict how third party code is using fork(): "Take cover, dead locks ahead!" :) A cooperative design of the C API with three function is my preferred way, too. PyOS_AfterForkParent() should take an argument to signal a failed fork() call.

    @AmauryForgeotdArc
    Copy link
    Mannequin

    @AmauryForgeotdArc AmauryForgeotdArc mannequin commented Nov 20, 2012

    2012/11/20 Christian Heimes <report@bugs.python.org>

    IFF we are going to walk the hard and rocky road of exception handling,
    then we are going to need at least four hooks and a register function that
    takres four callables as arguments: register(prepare, error, parent,
    child). Each prepare() call pushes an error handling onto a stack. In case
    of an exception in a prepare handler, the error stack is popped until all
    error handlers are called. This approach allows a prepare handler to
    actually prevent a fork() call from succeeding.

    FWIW, PyPy already has a notion of fork hooks:
    https://bitbucket.org/pypy/pypy/src/b4e4017909bac6c102fbc883ac8d2e42fa41553b/pypy/module/posix/interp_posix.py?at=default#cl-682

    Various subsystems (threads cleanup, import lock, threading.local...)
    register their hook functions.

    You may want to experiment from there :-)
    A new "atfork" module would be easy to implement.

    @sbt
    Copy link
    Mannequin

    @sbt sbt mannequin commented Nov 20, 2012

    IFF we are going to walk the hard and rocky road of exception handling,
    then we are going to need at least four hooks and a register function that
    takres four callables as arguments: register(prepare, error, parent,
    child). Each prepare() call pushes an error handling onto a stack. In case
    of an exception in a prepare handler, the error stack is popped until all
    error handlers are called. This approach allows a prepare handler to
    actually prevent a fork() call from succeeding.

    I think there are two main options if a prepare callback fails:

    1. The fork should not occur and the exception should be raised
    2. The fork should occur and the exception should be only be printed

    I favour option 1 since, if they want, users can always wrap their prepare callbacks with

    try:
    ...
    except:
    sys.excepthook(*sys.exc_info())

    With option 1 I don't see why error callbacks are necessary. Just unwind the stack of imaginary try...finally... clauses and let any exceptions propagate out using exception chaining if necessary. This is what pure-python-atfork.patch does. Note, however, that if the fork succeeds then any subsequent exception is only printed.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 20, 2012

    Amaury:
    PyPy doesn't handle exceptions in hooks. Is there a reason why PyPy goes for the simplistic approach?

    Richard:
    An error callback has the benefit that the API can notice the hooks that some error has occurred. We may not need it, though.

    I can think of six exception scenarios that must be handled:

    (1) exception in a prepare hook -> don't call the remaining prepare hooks, run all related parent hooks in FILO order, prevent fork() call
    (2) exception in parent hook during the handling of (1) -> print exception, continue with next parent hook
    (3) exception in fork() call -> run parent hooks in FILO order
    (4) exception in parent hook during the handling of (3) -> print exception, continue with next parent hook
    (5) exception in parent hook when fork() has succeeded -> print exception, continue with next parent hook
    (6) exception in child hook when fork() has succeeded -> print exception, continue with next child hook

    Do you agree?

    @amauryfa
    Copy link
    Member

    @amauryfa amauryfa commented Nov 20, 2012

    PyPy doesn't handle exceptions in hooks.
    Is there a reason why PyPy goes for the simplistic approach?

    Probably because nobody thought about it.
    At the moment, there is only one 'before', one 'parent' hook (so the FILO order is simple), and three 'child' hooks.
    And if the _PyImport_ReleaseLock call fails, you'd better not ignore the error...

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented Nov 20, 2012

    I think you are solving a non-problem if you want to expose exceptions from
    such hooks. Nobody needs it.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Nov 20, 2012

    I think you are solving a non-problem if you want to expose exceptions from
    such hooks. Nobody needs it.

    Agreed.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Nov 20, 2012

    Your suggestion is that the hooks are called as:

    for hook in hooks:
        try:
            hook()
        except:
            try:
                sys.excepthook(*sys.exc_info())
            except:
                pass

    That makes the implementation much easier. :)

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jan 12, 2013

    "The tempfile module has a specialized RNG that re-initialized the RNG after fork() by comparing os.getpid() to an instance variable every time the RNG is accessed. The check can be replaced with an afterfork callback."

    By the way, OpenSSL expects that its PRNG is reseed somehow (call RNG_add) after a fork. I wrote a patch for OpenSSL, but I don't remember if I sent it to OpenSSL.
    https://bitbucket.org/haypo/hasard/src/4a1be69a47eb1b2ec7ca95a341d4ca953a77f8c6/patches/openssl_rand_fork.patch?at=default

    Reseeding tempfile PRNG is useless (but spend CPU/memory/hang until we have enough entropy?) if the tempfile is not used after fork. I like the current approach.

    --

    I'm not saying that a new atfork module would not help, just that the specific case of tempfile should be discussed :-) I like the idea of a generic module to call code after fork.

    @vstinner vstinner changed the title Add an 'afterfork' module Add an 'atfork' module Jan 12, 2013
    @birkenfeld
    Copy link
    Member

    @birkenfeld birkenfeld commented Jan 13, 2013

    Might make sense to put this in atexit.atfork() to avoid small-module inflation?

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Jan 14, 2013

    Might make sense to put this in atexit.atfork() to avoid small-module inflation?

    It sounds strange to mix "at exit" and "at fork" in the same module.
    Both are very different.

    2013/1/13 Arfrever Frehtes Taifersar Arahesis <report@bugs.python.org>:

    Changes by Arfrever Frehtes Taifersar Arahesis <Arfrever.FTA@GMail.Com>:

    ----------
    nosy: +Arfrever


    Python tracker <report@bugs.python.org>
    <http://bugs.python.org/issue16500\>


    @malemburg
    Copy link
    Member

    @malemburg malemburg commented Jan 14, 2013

    On 13.01.2013 00:37, STINNER Victor wrote:

    By the way, OpenSSL expects that its PRNG is reseed somehow (call RNG_add) after a fork. I wrote a patch for OpenSSL, but I don't remember if I sent it to OpenSSL.
    https://bitbucket.org/haypo/hasard/src/4a1be69a47eb1b2ec7ca95a341d4ca953a77f8c6/patches/openssl_rand_fork.patch?at=default

    Apparently not, and according to this thread, they don't think
    this is an OpenSSL problem to solve:

    http://openssl.6102.n7.nabble.com/recycled-pids-causes-PRNG-to-repeat-td41669.html

    Note that you don't have to reseed the RNG just make sure that the
    two forks use different sequences. Simply adding some extra data
    in each process will suffice, e.g. by adding the PID of the new process
    to the RNG pool. This is certainly doable without any major CPU
    overhead :-)

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented Jan 14, 2013

    It sounds strange to mix "at exit" and "at fork" in the same module.
    Both are very different.

    That's true. The sys module would probably be the right place for both functionalities.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 21, 2013

    Richard, do you have time to get your patch ready for 3.4?

    @tiran tiran removed their assignment Oct 21, 2013
    @sbt
    Copy link
    Mannequin

    @sbt sbt mannequin commented Oct 21, 2013

    Richard, do you have time to get your patch ready for 3.4?

    Yes. But we don't seem to have concensus on how to handle exceptions. The main question is whether a failed prepare callback should prevent the fork from happenning, or just be printed.

    @DLitz
    Copy link
    Mannequin

    @DLitz DLitz mannequin commented Oct 21, 2013

    The main question is whether a failed prepare callback should prevent the fork from happenning

    Yes, I think an exception should prevent the fork from happening.

    It's fail-safe for the PRNG case (you can guarantee that a fork won't occur without properly re-seeding a PRNG), and it makes bugs easier to catch in unit testing.

    @tiran
    Copy link
    Member Author

    @tiran tiran commented Oct 21, 2013

    +1 for exception prevents fork

    @neologix
    Copy link
    Mannequin

    @neologix neologix mannequin commented Oct 21, 2013

    I have a couple random remarks:

    • now that FDs are non-inheritable by default, fork locks around
      subprocess and multiprocessing shouldn't be necessary anymore? What
      other use cases does the fork-lock have?
    • the current implementation keeps hard-references to the functions
      passed: so if one isn't careful, you can end up easily with a lot of
      objects kept alive just because of those references, which can be a
      problem
    • also, since we're not sure about the API, and it's mostly intended
      to be used for the interpreter/stdlib, how about making it private for
      now, or at least "provisional' (I think that's the term)?
    • I'm also +1 on exceptions in prepare hook preventing fork, but we'll
      need to play a bit with actual fork hooks to see if that's a
      reasonable approach

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 22, 2017

    API-wise, I went for the minimal route. This avoids any discussion of adding a separate module for a tiny functionality that is only going to be used by a couple of libraries (and probably no application code).

    Comparisons with atexit are not really relevant, IMO, since the use cases are very different.

    As for passing explicit arguments to the callable, people can use a lambda or functools.partial. I don't want to complicate the C implementation with matters that are not really important.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 22, 2017

    I agree that with lambdas and functools.partial the support of arguments is not needed. Unless someone has good reasons for supporting explicit passing of arguments I'm fine with your design.

    If the user code manually calls the fork() and (now deprecated) PyOS_AfterFork(), it runs "child" functions, but not "before" and "parent" functions. Is it worth to emit a runtime warning if the "before" or "parent" lists are not empty? Or even immediately exit a child with error?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 22, 2017

    I don't think exiting would be a good idea at all. I'm not sure about emitting a warning: the problem is that the people seeing the warning (plain users) can't do anything about it; worse, they probably will not even understand what it is about, and will get confused.

    The package maintainer should see a deprecation warning when compiling code using PyOS_AfterFork() (thanks to the Py_DEPRECATED attribute).

    Of course, I'll also update the docs to add a deprecation once the design is settled.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 22, 2017

    PyOS_AfterFork() is used by one code, but the "before" and "parent" handlers are registered by the other code (likely the code of other library). The author of the program that uses both libraries (the one that uses PyOS_AfterFork() and the one that registers handlers) can notice the warning and report the bug in the first library. I think silently skipping registered handlers would be worse.

    Let allow the user to control the behavior by setting the warnings filter. If the corresponding warnings are ignored, nothing happen, if they are errors, the child is exited, otherwise the warning message is output on stderr as for other warnings.

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 22, 2017

    If the corresponding warnings are ignored, nothing happen, if they are errors, the child is exited,

    Right now PyOS_AfterFork() doesn't return an error code. It is not obvious how the caller would react: simply print the error? raise a fatal error? Something else?

    The only third-party use of PyOS_AfterFork() I found is in uwsgi (and I'm not sure they're using it correctly, since I don't know if the parent process is using Python at all...).

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 27, 2017

    New changeset 346cbd3 by Antoine Pitrou in branch 'master':
    bpo-16500: Allow registering at-fork handlers (bpo-1715)
    346cbd3

    @pitrou pitrou closed this as completed May 27, 2017
    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 28, 2017

    Could you please update the documentation Antoine?

    • An entry in the C API section of Misc/NEWS.
    • The "What's New" document needs changes in multiple places: the os module improvement, C API additions and deprecation, and may be in porting guide.
    • Update references to PyOS_AfterFork() in the documentation and comments.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 28, 2017

    Can threading._after_fork() be rewritten with using the new API?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 28, 2017

    New changeset f7ecfac by Antoine Pitrou in branch 'master':
    Doc nits for bpo-16500 (bpo-1841)
    f7ecfac

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 28, 2017

    Le 28/05/2017 à 11:22, Serhiy Storchaka a écrit :

    Serhiy Storchaka added the comment:

    Can threading._after_fork() be rewritten with using the new API?

    It should be possible indeed. Let me see.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 28, 2017

    Can multiprocessing.util.register_after_fork() be rewritten with using the new API?

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 28, 2017

    Can multiprocessing.util.register_after_fork() be rewritten with using the new API?

    It wouldn't benefit much from it, and there might be timing issue given the comments in BaseProcess._bootstrap():

                old_process = _current_process
                _current_process = self
                try:
                    util._finalizer_registry.clear()
                    util._run_after_forkers()
                finally:
                    # delay finalization of the old process object until after
                    # _run_after_forkers() is executed
                    del old_process

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 28, 2017

    New changeset 4a8bcdf by Antoine Pitrou in branch 'master':
    bpo-16500: Use register_at_fork() in the threading module (bpo-1843)
    4a8bcdf

    @pitrou pitrou closed this as completed May 28, 2017
    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 28, 2017

    In PR 1834 Gregory proposes an alternate API:

        os.register_at_fork(*, before=None, after_in_parent=None, after_in_child=None)

    Maybe open a new issue for this?

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented May 28, 2017

    I'll just reuse this issue, I started that work just to try and clean up the new API and docs added in this one to be more modern. Thanks for the code review, good suggestions!

    @gpshead gpshead reopened this May 28, 2017
    @gpshead gpshead self-assigned this May 28, 2017
    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented May 29, 2017

    The one of potential advantages of the API proposed by Gregory is that os.register_at_fork() can be made atomic. Either register all callbacks, or do nothing in the case of error. But current proposed implementation is not atomic. If resizing of some list is failed due to MemoryError (or may be KeyboardInterrupt), some callbacks can already be registered.

    Is it worth to guarantee the atomicity of os.register_at_fork().

    @pitrou
    Copy link
    Member

    @pitrou pitrou commented May 29, 2017

    Le 29/05/2017 à 18:35, Serhiy Storchaka a écrit :

    Is it worth to guarantee the atomicity of os.register_at_fork().

    I don't think so, honestly.

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented May 29, 2017

    New changeset 163468a by Gregory P. Smith in branch 'master':
    bpo-16500: Don't use string constants for os.register_at_fork() behavior (bpo-1834)
    163468a

    @gpshead gpshead closed this as completed May 29, 2017
    @vstinner
    Copy link
    Member

    @vstinner vstinner commented May 29, 2017

    New changeset 163468a by Gregory P. Smith in branch 'master':
    bpo-16500: Don't use string constants for os.register_at_fork() behavior (bpo-1834)
    163468a

    Since there was a disagreement, FYI I also prefer this API. Just the
    API, I didn't look at the implementation in detail.

    Anyway, it's nice to see this very old issue finally fixed! It was
    opened 5 years ago!

    I also prefer to have a builtin registered callback in the random
    module instead of having a custom callback in the multiprocessing
    module to re-seed the RNG after fork!

    @gpshead
    Copy link
    Member

    @gpshead gpshead commented May 29, 2017

    This may also allow us to do something reasonable for http://bugs.python.org/issue6721 as well.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Sep 12, 2017

    New changeset a15d155 by Victor Stinner in branch 'master':
    bpo-31234: Enhance test_thread.test_forkinthread() (bpo-3516)
    a15d155

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Sep 12, 2017

    New changeset bcf042f by Victor Stinner (Miss Islington (bot)) in branch '3.6':
    [3.6] bpo-31234: Enhance test_thread.test_forkinthread() (GH-3516) (bpo-3519)
    bcf042f

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 extension-modules interpreter-core type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    8 participants