Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Many multiprocessing tests are silently skipped since 3.9 #89205

Closed
serhiy-storchaka opened this issue Aug 29, 2021 · 13 comments
Closed

Many multiprocessing tests are silently skipped since 3.9 #89205

serhiy-storchaka opened this issue Aug 29, 2021 · 13 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 45042
Nosy @pitrou, @vstinner, @ambv, @serhiy-storchaka, @applio, @miss-islington, @sobolevn
PRs
  • bpo-45042: Now test classes decorated with requires_hashdigest are not skipped #28060
  • [3.10] bpo-45042: Now test classes decorated with requires_hashdigest are not skipped (GH-28060) #28168
  • [3.9] bpo-45042: Now test classes decorated with `requires_hashdigest… #28169
  • bpo-45052: Unskips a failing test_shared_memory_basics test #28182
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-09-05.06:42:38.629>
    created_at = <Date 2021-08-29.11:01:30.705>
    labels = ['type-bug', 'tests', '3.9', '3.10', '3.11']
    title = 'Many multiprocessing tests are silently skipped since 3.9'
    updated_at = <Date 2021-09-20.07:49:54.187>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2021-09-20.07:49:54.187>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-09-05.06:42:38.629>
    closer = 'serhiy.storchaka'
    components = ['Tests']
    creation = <Date 2021-08-29.11:01:30.705>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45042
    keywords = ['patch', '3.9regression']
    message_count = 13.0
    messages = ['400521', '400587', '400588', '400590', '400594', '400597', '400605', '400612', '400650', '401060', '401061', '401072', '402204']
    nosy_count = 7.0
    nosy_names = ['pitrou', 'vstinner', 'lukasz.langa', 'serhiy.storchaka', 'davin', 'miss-islington', 'sobolevn']
    pr_nums = ['28060', '28168', '28169', '28182']
    priority = 'high'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue45042'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @serhiy-storchaka
    Copy link
    Member Author

    Here is a list of multiprocessing tests which are run in 3.8 but are not found in 3.9+:

    OtherTest.test_answer_challenge_auth_failure
    OtherTest.test_deliver_challenge_auth_failure
    TestInitializers.test_manager_initializer
    TestInitializers.test_pool_initializer
    TestSyncManagerTypes.test_array
    TestSyncManagerTypes.test_barrier
    TestSyncManagerTypes.test_bounded_semaphore
    TestSyncManagerTypes.test_condition
    TestSyncManagerTypes.test_dict
    TestSyncManagerTypes.test_event
    TestSyncManagerTypes.test_joinable_queue
    TestSyncManagerTypes.test_list
    TestSyncManagerTypes.test_lock
    TestSyncManagerTypes.test_namespace
    TestSyncManagerTypes.test_pool
    TestSyncManagerTypes.test_queue
    TestSyncManagerTypes.test_rlock
    TestSyncManagerTypes.test_semaphore
    TestSyncManagerTypes.test_value
    WithManagerTestBarrier.test_abort
    WithManagerTestBarrier.test_abort_and_reset
    WithManagerTestBarrier.test_action
    WithManagerTestBarrier.test_barrier
    WithManagerTestBarrier.test_barrier_10
    WithManagerTestBarrier.test_default_timeout
    WithManagerTestBarrier.test_reset
    WithManagerTestBarrier.test_single_thread
    WithManagerTestBarrier.test_thousand
    WithManagerTestBarrier.test_timeout
    WithManagerTestBarrier.test_wait_return
    WithManagerTestCondition.test_notify
    WithManagerTestCondition.test_notify_all
    WithManagerTestCondition.test_notify_n
    WithManagerTestCondition.test_timeout
    WithManagerTestCondition.test_wait_result
    WithManagerTestCondition.test_waitfor
    WithManagerTestCondition.test_waitfor_timeout
    WithManagerTestContainers.test_dict
    WithManagerTestContainers.test_dict_iter
    WithManagerTestContainers.test_dict_proxy_nested
    WithManagerTestContainers.test_list
    WithManagerTestContainers.test_list_iter
    WithManagerTestContainers.test_list_proxy_in_list
    WithManagerTestContainers.test_namespace
    WithManagerTestEvent.test_event
    WithManagerTestLock.test_lock
    WithManagerTestLock.test_lock_context
    WithManagerTestLock.test_rlock
    WithManagerTestManagerRestart.test_rapid_restart
    WithManagerTestMyManager.test_mymanager
    WithManagerTestMyManager.test_mymanager_context
    WithManagerTestMyManager.test_mymanager_context_prestarted
    WithManagerTestPool.test_apply
    WithManagerTestPool.test_async
    WithManagerTestPool.test_async_timeout
    WithManagerTestPool.test_context
    WithManagerTestPool.test_empty_iterable
    WithManagerTestPool.test_enter
    WithManagerTestPool.test_imap
    WithManagerTestPool.test_imap_handle_iterable_exception
    WithManagerTestPool.test_imap_unordered
    WithManagerTestPool.test_imap_unordered_handle_iterable_exception
    WithManagerTestPool.test_make_pool
    WithManagerTestPool.test_map
    WithManagerTestPool.test_map_async
    WithManagerTestPool.test_map_async_callbacks
    WithManagerTestPool.test_map_chunksize
    WithManagerTestPool.test_map_handle_iterable_exception
    WithManagerTestPool.test_map_no_failfast
    WithManagerTestPool.test_map_unplicklable
    WithManagerTestPool.test_release_task_refs
    WithManagerTestPool.test_resource_warning
    WithManagerTestPool.test_starmap
    WithManagerTestPool.test_starmap_async
    WithManagerTestPool.test_terminate
    WithManagerTestPool.test_traceback
    WithManagerTestPool.test_wrapped_exception
    WithManagerTestQueue.test_closed_queue_put_get_exceptions
    WithManagerTestQueue.test_fork
    WithManagerTestQueue.test_get
    WithManagerTestQueue.test_no_import_lock_contention
    WithManagerTestQueue.test_put
    WithManagerTestQueue.test_qsize
    WithManagerTestQueue.test_queue_feeder_donot_stop_onexc
    WithManagerTestQueue.test_queue_feeder_on_queue_feeder_error
    WithManagerTestQueue.test_task_done
    WithManagerTestQueue.test_timeout
    WithManagerTestRemoteManager.test_remote
    WithManagerTestSemaphore.test_bounded_semaphore
    WithManagerTestSemaphore.test_semaphore
    WithManagerTestSemaphore.test_timeout
    WithProcessesTestManagerRestart.test_rapid_restart
    WithProcessesTestPicklingConnections.test_access
    WithProcessesTestPicklingConnections.test_pickling
    WithProcessesTestSharedMemory.test_shared_memory_ShareableList_basics
    WithProcessesTestSharedMemory.test_shared_memory_ShareableList_pickling
    WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_basics
    WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_reuses_resource_tracker
    WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryServer_ignores_sigint
    WithProcessesTestSharedMemory.test_shared_memory_across_processes
    WithProcessesTestSharedMemory.test_shared_memory_basics
    WithProcessesTestSharedMemory.test_shared_memory_cleaned_after_process_termination
    WithThreadsTestManagerRestart.test_rapid_restart

    @serhiy-storchaka serhiy-storchaka added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels Aug 29, 2021
    @sobolevn
    Copy link
    Member

    Looks like that this happens because tests classes are decorated with @hashlib_helper.requires_hashdigest('md5'):

    @hashlib_helper.requires_hashdigest('md5')
    class OtherTest(unittest.TestCase):
    # TODO: add more tests for deliver/answer challenge.
    def test_deliver_challenge_auth_failure(self):

    Tests above run fine without this decorator.
    It was added in https://bugs.python.org/issue17258

    The problem is that `` was treating a decorated entity as a function, not a class.

    @functools.wraps(func)
    But, it was decorating classes as well.
    So, it needs to changed to respect classes.

    I will make a PR shortly.

    @pitrou
    Copy link
    Member

    pitrou commented Aug 30, 2021

    Wow. I hope this didn't hide any regression :-(

    @sobolevn
    Copy link
    Member

    Looks like it did, two newly-unignored tests are failing:
    #28060 (comment)

    пн, 30 авг. 2021 г. в 15:16, Antoine Pitrou <report@bugs.python.org>:

    Antoine Pitrou <pitrou@free.fr> added the comment:

    Wow. I hope this didn't hide any regression :-(

    ----------


    Python tracker <report@bugs.python.org>
    <https://bugs.python.org/issue45042\>


    @serhiy-storchaka
    Copy link
    Member Author

    Thank you Nikita.

    The difference between the original and the proposed in PR 28060 code is that in the original code the conditional was tested at the testing time, while in the proposed code it is tested at the loading time. I do not know what effect it causes on tests. If it does not matter, the code can be simpler:

       return unittest.skipIf(should_be_skipped,
                              f"hash digest '{digestname}' is not available.")

    But if it matters, there are two options:

    1. In requires_hashdigest() raise exception if func is a class. Remove the decorator from classes and add it to test and setup methods or just to the setUpClass() classmethod.

    2. In requires_hashdigest() check if func is a class, and if it is true, patch its test and setup methods (or just add a decorated setUpClass() classmethod).

    @serhiy-storchaka
    Copy link
    Member Author

    Here is the implementation of the second option:

    def requires_hashdigest(digestname, openssl=None, usedforsecurity=True):
        def decorator(func):
            if isinstance(func, type):
                setUpClass = func.__dict__.get('setUpClass')
                if setUpClass is None:
                    def setUpClass(cls):
                        super(func, cls).setUpClass()
                    setUpClass.__qualname__ = func.__qualname__ + '.setUpClass'
                    setUpClass.__module__ = func.__module__
                else:
                    setUpClass = setUpClass.__func__
                setUpClass = classmethod(decorator(setUpClass))
                func.setUpClass = setUpClass
                return func
            @functools.wraps(func)
            def wrapper(*args, **kwargs):
                try:
                    if openssl and _hashlib is not None:
                        _hashlib.new(digestname, usedforsecurity=usedforsecurity)
                    else:
                        hashlib.new(digestname, usedforsecurity=usedforsecurity)
                except ValueError:
                    raise unittest.SkipTest(
                        f"hash digest '{digestname}' is not available."
                    )
                return func(*args, **kwargs)
            return wrapper
        return decorator

    @sobolevn
    Copy link
    Member

    Serhiy, yes, you are right. I guess, it is safe to assume that load-time/test-time might make a difference, especially with _hashlib.

    I've commited your suggestion, thanks a lot for your help!

    @ambv
    Copy link
    Contributor

    ambv commented Aug 30, 2021

    FYI, there seem to be two Windows-specific regressions since the tests were unintentionally disabled, namely test_shared_memory_basics and test_checksum_fodder.

    Following Serhiy's advice, I elect to have those tests skipped for now on Windows and fix them through separate issues. We will be releasing 3.9.7 according to schedule today.

    @sobolevn
    Copy link
    Member

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset dd7b816 by Nikita Sobolev in branch 'main':
    bpo-45042: Now test classes decorated with requires_hashdigest are not skipped (GH-28060)
    dd7b816

    @miss-islington
    Copy link
    Contributor

    New changeset e5976dd by Miss Islington (bot) in branch '3.10':
    bpo-45042: Now test classes decorated with requires_hashdigest are not skipped (GH-28060)
    e5976dd

    @serhiy-storchaka
    Copy link
    Member Author

    New changeset ab58269 by Serhiy Storchaka in branch '3.9':
    [3.9] bpo-45042: Now test classes decorated with requires_hashdigest are not skipped (GH-28060) (GH-28169)
    ab58269

    @vstinner
    Copy link
    Member

    See also bpo-45128 (fixed): "test_multiprocessing_fork fails if run sequentially after test_genericalias and test_logging".

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants