Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NSS: close files after mmap #560

Conversation

ChrisKowalczyk
Copy link

The files in MC cache folder were initialized by SSSD on startup, and mapped by using mmap function. due to the fact that they weren't closed afterwards, their File descriptors were still marker alive but marked as 'Deleted'.
This was noticed by a customer of SUSE, see more details here: https://bugzilla.suse.com/show_bug.cgi?id=1080156

@centos-ci
Copy link

Can one of the admins verify this patch?

1 similar comment
@centos-ci
Copy link

Can one of the admins verify this patch?

@fidencio
Copy link
Contributor

ok to test

@fidencio
Copy link
Contributor

retest this please

@fidencio
Copy link
Contributor

@ChrisKowalczyk,

Firstly, sorry that I'm not going to review the patch itself (I'll leave it to someone else).

What I'd like to bring up here is that a bunch of tests are failing due to the change:

=========================================== FAILURES ============================================
_________________________________ test_add_remove_add_file_user _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 534, in test_add_remove_add_file_user
    assert res == NssReturnCode.NOTFOUND
AssertionError: assert 1 == 0
 +  where 0 = NssReturnCode.NOTFOUND
______________________________________ test_mod_user_shell ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 561, in test_mod_user_shell
    check_user(moduser)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 375, in check_user
    assert found_user == exp_user
AssertionError: assert {'dir': '/hom... 'user1', ...} == {'dir': '/home... 'user1', ...}
  Omitting 6 identical items, use -vv to show
  Differing items:
  {'shell': '/bin/bash'} != {'shell': '/bin/zsh'}
  Full diff:
  - {'dir': u'/home/user1',
  ?         -
  + {'dir': '/home/user1',...
  
  ...Full output truncated (16 lines hidden), use '-vv' to show
______________________________________ test_user_no_shell _______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 597, in test_user_no_shell
    check_user(incomplete_user_setup(setup_pw_with_canary, 'shell', ''))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 375, in check_user
    assert found_user == exp_user
AssertionError: assert {'dir': '/hom... 'user1', ...} == {'dir': '/home... 'user1', ...}
  Omitting 6 identical items, use -vv to show
  Differing items:
  {'shell': '/bin/bash'} != {'shell': ''}
  Full diff:
  - {'dir': u'/home/user1',
  ?         -
  + {'dir': '/home/user1',...
  
  ...Full output truncated (14 lines hidden), use '-vv' to show
_______________________________________ test_user_no_dir ________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 605, in test_user_no_dir
    check_user(incomplete_user_setup(setup_pw_with_canary, 'dir', '/'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 375, in check_user
    assert found_user == exp_user
AssertionError: assert {'dir': '/hom... 'user1', ...} == {'dir': '/', '... 'user1', ...}
  Omitting 6 identical items, use -vv to show
  Differing items:
  {'dir': '/home/user1'} != {'dir': '/'}
  Full diff:
  - {'dir': u'/home/user1',
  + {'dir': '/',
  -  'gecos': u'User for tests',...
  
  ...Full output truncated (14 lines hidden), use '-vv' to show
______________________________________ test_user_no_gecos _______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 613, in test_user_no_gecos
    check_user(incomplete_user_setup(setup_pw_with_canary, 'gecos', ''))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 375, in check_user
    assert found_user == exp_user
AssertionError: assert {'dir': '/hom... 'user1', ...} == {'dir': '/home... 'user1', ...}
  Omitting 6 identical items, use -vv to show
  Differing items:
  {'gecos': 'User for tests'} != {'gecos': ''}
  Full diff:
  - {'dir': u'/home/user1',
  ?         -
  + {'dir': '/home/user1',...
  
  ...Full output truncated (14 lines hidden), use '-vv' to show
___________________________________ test_incomplete_user_fail ___________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 641, in test_incomplete_user_fail
    assert res == NssReturnCode.NOTFOUND
AssertionError: assert 1 == 0
 +  where 0 = NssReturnCode.NOTFOUND
______________________________________ test_getgrnam_ghost ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 857, in test_getgrnam_ghost
    assert res == NssReturnCode.NOTFOUND
AssertionError: assert 1 == 0
 +  where 0 = NssReturnCode.NOTFOUND
________________________________ test_getgrnam_add_remove_ghosts ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 968, in test_getgrnam_add_remove_ghosts
    assert res == NssReturnCode.NOTFOUND
AssertionError: assert 1 == 0
 +  where 0 = NssReturnCode.NOTFOUND
________________________________ test_proxy_to_files_domain_only ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 1059, in test_proxy_to_files_domain_only
    assert user == local_user1
AssertionError: assert {'dir': '/hom... 'user1', ...} == {'dir': '/home... 'user1', ...}
  Omitting 3 identical items, use -vv to show
  Differing items:
  {'passwd': 'x'} != {'passwd': '*'}
  {'gid': 20001} != {'gid': 10009}
  {'gecos': 'User for tests'} != {'gecos': 'user1'}
  {'uid': 10001} != {'uid': 10009}
  Full diff:...
  
  ...Full output truncated (24 lines hidden), use '-vv' to show
___________________________________ test_disable_files_domain ___________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_files_provider.py", line 1088, in test_disable_files_domain
    assert res != NssReturnCode.SUCCESS
AssertionError: assert 1 != 1
 +  where 1 = NssReturnCode.SUCCESS
______________________________________ test_sanity_rfc2307 ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 323, in test_sanity_rfc2307
    ent.assert_each_passwd_by_name(passwd_pattern)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'gecos' mismatch: '1001' != 'User for tests'
____________________________________ test_sanity_rfc2307_bis ____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 355, in test_sanity_rfc2307_bis
    ent.assert_each_passwd_by_name(passwd_pattern)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'gecos' mismatch: '1001' != 'User for tests'
________________________________ test_refresh_after_cleanup_task ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 429, in test_refresh_after_cleanup_task
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gecos' mismatch: '1001' != 'User for tests'
____________________________ test_update_ts_cache_after_cleanup_task ____________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 472, in test_update_ts_cache_after_cleanup_task
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gecos' mismatch: '1001' != 'User for tests'
________________________________ test_user_2307bis_nested_groups ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 778, in test_user_2307bis_nested_groups
    gid=primary_gid))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 2001 != 20001
____________________________________ test_add_user_to_group _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 875, in test_add_user_to_group
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=2001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 2001 != 20001
__________________________________ test_remove_user_from_group __________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 903, in test_remove_user_from_group
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=2001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 2001 != 20001
______________________________ test_remove_user_from_nested_group _______________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 960, in test_remove_user_from_nested_group
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=2001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 2001 != 20001
____________________________________ test_zero_nesting_level ____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1057, in test_zero_nesting_level
    assert sorted(grp_list) == sorted(["primarygroup", "parentgroup"])
AssertionError: assert ['20001', 'parentgroup'] == ['parentgroup', 'primarygroup']
  At index 0 diff: '20001' != 'parentgroup'
  Full diff:
  - ['20001', 'parentgroup']
  + ['parentgroup', 'primarygroup']
_______________________________________ test_nss_filters ________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1108, in test_nss_filters
    ent.assert_each_passwd_by_name(passwd_pattern)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'gecos' mismatch: '1001' != 'User for tests'
____________________________________ test_nss_filters_cached ____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1191, in test_nss_filters_cached
    ent.assert_each_passwd_by_name(passwd_pattern)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'gecos' mismatch: '1001' != 'User for tests'
_____________________________ test_ldap_auto_private_groups_direct ______________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1264, in test_ldap_auto_private_groups_direct
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=1001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 1001 != 20001
____________________________ test_ldap_auto_private_groups_conflict _____________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1357, in test_ldap_auto_private_groups_conflict
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=1001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 1001 != 20001
__________________________ test_ldap_auto_private_groups_direct_no_gid __________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1419, in test_ldap_auto_private_groups_direct_no_gid
    ent.assert_passwd_by_name("user1", dict(name="user1", uid=1001, gid=1001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gid' mismatch: 1001 != 20001
_____________________________ test_rename_incomplete_group_same_dn ______________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1531, in test_rename_incomplete_group_same_dn
    assert sorted(grp_list) == sorted(["pvt_user1", "user1_group1"])
AssertionError: assert ['20001', 'user1_group1'] == ['pvt_user1', 'user1_group1']
  At index 0 diff: '20001' != 'pvt_user1'
  Full diff:
  - ['20001', 'user1_group1']
  ?   ^^^^
  + ['pvt_user1', 'user1_group1']
  ?   ^^^^^^^^
___________________________ test_rename_incomplete_group_rdn_changed ____________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ldap.py", line 1572, in test_rename_incomplete_group_rdn_changed
    assert sorted(grp_list) == sorted(["user1_private", "group1"])
AssertionError: assert ['20001', 'group1'] == ['group1', 'user1_private']
  At index 0 diff: '20001' != 'group1'
  Full diff:
  - ['20001', 'group1']
  + ['group1', 'user1_private']
_________________________________________ test_getpwnam _________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 245, in test_getpwnam
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gecos' mismatch: '1001' != 'User for tests'
_____________________________________ test_getpwnam_with_mc _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 325, in test_getpwnam_with_mc
    test_getpwnam(ldap_conn, sanity_rfc2307)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 245, in test_getpwnam
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 217, in assert_passwd_by_name
    assert not d, d
AssertionError: 'gecos' mismatch: '1001' != 'User for tests'
_________________________________ test_getgrnam_simple_with_mc __________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 353, in test_getgrnam_simple_with_mc
    test_getgrnam_simple(ldap_conn, sanity_rfc2307)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 331, in test_getgrnam_simple
    ent.assert_group_by_name("group1", dict(name="group1", gid=2001))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 375, in assert_group_by_name
    assert False, err
AssertionError: 'getgrnam(): name not found: group1'
_______________________________ test_getgrnam_membership_with_mc ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 403, in test_getgrnam_membership_with_mc
    test_getgrnam_membership(ldap_conn, sanity_rfc2307)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 359, in test_getgrnam_membership
    dict(mem=ent.contains_only("user1", "user11", "user21")))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 375, in assert_group_by_name
    assert False, err
AssertionError: 'getgrnam(): name not found: group1'
____________________________________ test_initgroups_with_mc ____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 435, in test_initgroups_with_mc
    test_initgroups(ldap_conn, sanity_rfc2307)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 419, in test_initgroups
    assert_user_gids_equal('user1', [2000, 2001])
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 407, in assert_user_gids_equal
    (res, errno, gids) = sssd_id.get_user_gids(user)
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/sssd_id.py", line 85, in get_user_gids
    pwd_user = pwd.getpwnam(user)
KeyError: 'getpwnam(): name not found: user1'
__________________________ test_invalidation_of_gids_after_initgroups ___________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 556, in test_invalidation_of_gids_after_initgroups
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 215, in assert_passwd_by_name
    assert False, err
AssertionError: 'getpwnam(): name not found: user1'
_________________________ test_initgroups_without_change_in_membership __________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 600, in test_initgroups_without_change_in_membership
    run_simple_test_with_initgroups()
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 519, in run_simple_test_with_initgroups
    gecos='1001', shell='/bin/bash'))
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 215, in assert_passwd_by_name
    assert False, err
AssertionError: 'getpwnam(): name not found: user1'
________________________________________ test_removed_mc ________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 880, in test_removed_mc
    pwd.getpwnam('user1')
  File "/usr/lib/python2.7/site-packages/_pytest/python_api.py", line 611, in __exit__
    fail(self.message)
  File "/usr/lib/python2.7/site-packages/_pytest/outcomes.py", line 85, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: DID NOT RAISE <type 'exceptions.KeyError'>
_____________________________________ test_mc_zero_timeout ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_memory_cache.py", line 913, in test_mc_zero_timeout
    pwd.getpwnam('user1')
  File "/usr/lib/python2.7/site-packages/_pytest/python_api.py", line 611, in __exit__
    fail(self.message)
  File "/usr/lib/python2.7/site-packages/_pytest/outcomes.py", line 85, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: DID NOT RAISE <type 'exceptions.KeyError'>
___________________________________________ test_none ___________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 267, in test_none
    dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 345, in assert_passwd
    assert not d, d
AssertionError: name retrieval mismatch: user 'user1' mismatch: 'shell' mismatch: '/bin/sh1' != '/bin/bash'
_________________________________________ test_all_nam __________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 297, in test_all_nam
    shell=config.SESSION_RECORDING_SHELL),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_________________________________________ test_all_uid __________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 311, in test_all_uid
    shell=config.SESSION_RECORDING_SHELL),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
________________________________________ test_some_empty ________________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 349, in test_some_empty
    dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 345, in assert_passwd
    assert not d, d
AssertionError: name retrieval mismatch: user 'user1' mismatch: 'shell' mismatch: '/bin/sh1' != '/bin/bash'
______________________________________ test_some_users_nam ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 378, in test_some_users_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
______________________________________ test_some_users_uid ______________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 390, in test_some_users_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
________________________________ test_some_users_overridden_uid _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 454, in test_some_users_overridden_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_____________________________________ test_some_groups1_nam _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 549, in test_some_groups1_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_____________________________________ test_some_groups1_uid _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 561, in test_some_groups1_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_____________________________________ test_some_groups2_nam _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 586, in test_some_groups2_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_____________________________________ test_some_groups2_uid _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 599, in test_some_groups2_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_____________________________________ test_some_groups3_nam _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 622, in test_some_groups3_nam
    shell=config.SESSION_RECORDING_SHELL),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/sh1' != '/bin/bash'
_____________________________________ test_some_groups3_uid _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 633, in test_some_groups3_uid
    shell=config.SESSION_RECORDING_SHELL),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/sh1' != '/bin/bash'
_____________________________________ test_some_groups4_nam _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 657, in test_some_groups4_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_____________________________________ test_some_groups4_uid _____________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 669, in test_some_groups4_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
_______________________________ test_some_groups_overridden1_nam ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 717, in test_some_groups_overridden1_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden1_uid ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 731, in test_some_groups_overridden1_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden2_nam ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 782, in test_some_groups_overridden2_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden2_uid ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 796, in test_some_groups_overridden2_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden3_nam ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 847, in test_some_groups_overridden3_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden3_uid ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 861, in test_some_groups_overridden3_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden4_nam ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 912, in test_some_groups_overridden4_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
_______________________________ test_some_groups_overridden4_uid ________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 926, in test_some_groups_overridden4_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
------------------------------------- Captured stdout setup -------------------------------------
SSSD needs to be restarted for the changes to take effect.
________________________________ test_some_users_and_groups_nam _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 972, in test_some_users_and_groups_nam
    user4=dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 292, in assert_each_passwd_by_name
    assert not d, d
AssertionError: user 'user1' mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
________________________________ test_some_users_and_groups_uid _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_session_recording.py", line 986, in test_some_users_and_groups_uid
    1004: dict(name="user4", uid=1004, shell="/bin/sh4"),
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/ent.py", line 301, in assert_each_passwd_by_uid
    assert not d, d
AssertionError: user 1001 mismatch: 'shell' mismatch: '/bin/false' != '/bin/bash'
__________________________________ test_user_show_basic_sanity __________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_sssctl.py", line 218, in test_user_show_basic_sanity
    assert output.find("Name: user1") != -1
AssertionError: assert -1 != -1
 +  where -1 = <built-in method find of unicode object at 0x7fb220423c90>('Name: user1')
 +    where <built-in method find of unicode object at 0x7fb220423c90> = 'User user1 is not present in cache.\n'.find
__________________________________ test_user_update_same_attrs __________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 568, in test_user_update_same_attrs
    assert_diff_attrval(ts_attrs, old_ts_attrs, "originalModifyTimestamp")
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 245, in assert_diff_attrval
    assert adict1.get(attr_name) is not None and \
AssertionError: assert ('20180425153409Z' is not None and '20180425153409Z' != '20180425153409Z')
 +  where '20180425153409Z' = <built-in method get of dict object at 0x7fb2202cf910>('originalModifyTimestamp')
 +    where <built-in method get of dict object at 0x7fb2202cf910> = {'dataExpireTimestamp': '1', 'originalModifyTimestamp': '20180425153409Z'}.get
 +  and   '20180425153409Z' = <built-in method get of dict object at 0x7fb2202cf910>('originalModifyTimestamp')
 +    where <built-in method get of dict object at 0x7fb2202cf910> = {'dataExpireTimestamp': '1', 'originalModifyTimestamp': '20180425153409Z'}.get
 +  and   '20180425153409Z' = <built-in method get of dict object at 0x7fb2205cc398>('originalModifyTimestamp')
 +    where <built-in method get of dict object at 0x7fb2205cc398> = {'dataExpireTimestamp': '1524675849', 'originalModifyTimestamp': '20180425153409Z'}.get
__________________________________ test_user_update_diff_attrs __________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 592, in test_user_update_diff_attrs
    assert_diff_attrval(sysdb_attrs, old_sysdb_attrs, "dataExpireTimestamp")
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 245, in assert_diff_attrval
    assert adict1.get(attr_name) is not None and \
AssertionError: assert ('1524675853' is not None and '1524675853' != '1524675853')
 +  where '1524675853' = <built-in method get of dict object at 0x7fb220494e88>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb220494e88> = {'dataExpireTimestamp': '1524675853', 'originalModifyTimestamp': '20180425153412Z'}.get
 +  and   '1524675853' = <built-in method get of dict object at 0x7fb220494e88>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb220494e88> = {'dataExpireTimestamp': '1524675853', 'originalModifyTimestamp': '20180425153412Z'}.get
 +  and   '1524675853' = <built-in method get of dict object at 0x7fb220320e88>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb220320e88> = {'dataExpireTimestamp': '1524675853', 'originalModifyTimestamp': '20180425153412Z'}.get
_________________________________ test_user_2307bis_delete_user _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 616, in test_user_2307bis_delete_user
    pwd.getpwnam("user1")
  File "/usr/lib/python2.7/site-packages/_pytest/python_api.py", line 611, in __exit__
    fail(self.message)
  File "/usr/lib/python2.7/site-packages/_pytest/outcomes.py", line 85, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: DID NOT RAISE <type 'exceptions.KeyError'>
________________________________ test_sss_cache_invalidate_user _________________________________
Traceback (most recent call last):
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 649, in test_sss_cache_invalidate_user
    assert_diff_attrval(ts_attrs, sysdb_attrs, "dataExpireTimestamp")
  File "/home/brq/ffidenci/src/upstream/sssd/src/tests/intg/test_ts_cache.py", line 245, in assert_diff_attrval
    assert adict1.get(attr_name) is not None and \
AssertionError: assert ('1' is not None and '1' != '1')
 +  where '1' = <built-in method get of dict object at 0x7fb2202cda28>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb2202cda28> = {'dataExpireTimestamp': '1', 'originalModifyTimestamp': '20180425153419Z'}.get
 +  and   '1' = <built-in method get of dict object at 0x7fb2202cda28>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb2202cda28> = {'dataExpireTimestamp': '1', 'originalModifyTimestamp': '20180425153419Z'}.get
 +  and   '1' = <built-in method get of dict object at 0x7fb2202c7050>('dataExpireTimestamp')
 +    where <built-in method get of dict object at 0x7fb2202c7050> = {'dataExpireTimestamp': '1', 'originalModifyTimestamp': '20180425153419Z'}.get
============================ 65 failed, 194 passed in 602.56 seconds ============================
make[1]: *** [Makefile:1040: intgcheck-installed] Error 1
make[1]: Leaving directory '/home/brq/ffidenci/src/upstream/sssd/x86_64/intg/bld/src/tests/intg'
make: *** [Makefile:33943: intgcheck-run] Error 2

Please, in order to reproduce it in your own machine, run:

make intgcheck-run

@ChrisKowalczyk
Copy link
Author

@fidencio,
I'm surprised it caused any problems (I'd assume everybody says it ;) ). I'll have a look at the tests and provide an update. Thank you for the quick feedback!

@mzidek-gh
Copy link
Contributor

Hi,
I do not know what issue is this patch trying to solve (I do not have Suse bugzilla account and I get Bugzilla – Access Denied), but the patch is not correct.

We can not close the mmap cache file descriptor after the memory cache context is initialized. The reason why the mmap context keeps the FD open is so that it does not need to create a new FD every time when we do some operation on the mmap cache. So the fact that the sssd_nss responder is keeping the FD open in the context is by desing.

I am not sure why the file is marked as deleted, but note that the mmap cache context is created when the sssd_nss responder is starting up, so when you manually delete the files (for example using the rm command) SSSD needs to be restarted, the old FD will be closed and new one opened. Not restarting SSSD after the mmap cache was removed with "rm" will make the mmap cache not working.

Michal

Copy link
Contributor

@simo5 simo5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Closing the FD can be done, but then you have to do more than that, see comments.

@@ -185,6 +185,9 @@ static errno_t sss_nss_mc_init_ctx(const char *name,
ret = 0;

done:
if (ctx->fd) {
close(ctx->fd);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the very least you must set ctx->fd to -1 after a close, or you'll cause trouble later (closing unrelated files if the context is freed).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are going to always close the file, which is a fine choice, then I would expect this patch to actually remove the fd element from the struct and use a local variable instead so that no manipulation of the fd is expected in sss_nss_mc_destroy_ctx() at all.

@jhrozek
Copy link
Contributor

jhrozek commented Apr 25, 2018

I wonder if the caches were deleted because someone called sss_cache?

@mzidek-gh
Copy link
Contributor

@ChrisKowalczyk Hmm.. i thought we are using the fd in the memory cache operations, but it does not seem to be the case, so yes we do not need the fd, just the pointer to the mmaped region. Forget what I said and follow @simo5 's advice, sorry for confusion :)

@jhrozek Right that is also one of the possible reasons, sss_cache can mark the cache as "recycled" and delete the file, the clients learn about this when they do next request to the cache and reload the new file.

@ChrisKowalczyk
Copy link
Author

Thanks for all the comments and quick feedback! Sorry for sharing a closed bug report, I will prepare some better description of this scenario. I will also make sure that the tests are not failing.
@mzidek-rh , yes, I think your first comment was true for the second place we use mmap in sssd. I will follow @simo5 advice.
@jhrozek , I think that sss_cache was not the case, and this happened just after starting sssd. But as I mentioned, I will double check everything, update PR and provide better description and logs.

@alexey-tikhonov
Copy link
Member

@ChrisKowalczyk, do you plan to work on this PR?
Or may I take over?

@alexey-tikhonov
Copy link
Member

alexey-tikhonov commented May 12, 2020

@ChrisKowalczyk Hmm.. i thought we are using the fd in the memory cache operations, but it does not seem to be the case, so yes we do not need the fd

Actually it seems we need open fd: see https://pagure.io/SSSD/sssd/issue/2726 (#3767)

Also to be consistent NSS responder's handling of fd should be updated symmetrically.

As I don't know original intention (what problem this PR was intended to solve) I can't amend this PR properly.

Thus I will close this.

Please open a new issue or PR with a better description if there is any actual problem to solve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants