Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add additional asserts to help investigate test_win_pkg failure #49809

Merged
merged 10 commits into from Oct 13, 2018
Merged

Add additional asserts to help investigate test_win_pkg failure #49809

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
18e9a28
Add additional asserts to help investigate test_win_pkg failure
Ch3LL Sep 27, 2018
9dce5cc
Fix the assertion message order
Ch3LL Oct 4, 2018
1005b91
Debug win_pkg
dwoz Oct 4, 2018
2cd4058
Merge pull request #4 from dwoz/win_pkg_flaky
Ch3LL Oct 5, 2018
c386826
Log results of cache_dir for now
dwoz Oct 5, 2018
fd5e842
Merge pull request #5 from dwoz/win_pkg_flaky
Ch3LL Oct 5, 2018
e5ea3f1
Try running fsync to avoid fs caching race condition
dwoz Oct 6, 2018
75809b4
Merge pull request #6 from dwoz/win_pkg_flaky
Ch3LL Oct 8, 2018
beaab2a
Fix win_pkg test
dwoz Oct 11, 2018
f795b60
Merge pull request #7 from dwoz/win_pkg_flaky
Ch3LL Oct 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 4 additions & 1 deletion salt/fileserver/__init__.py
View file
Open in desktop
Expand Up @@ -131,7 +131,10 @@ def check_file_list_cache(opts, form, list_cache, w_lock):
if os.path.exists(list_cache):
# calculate filelist age is possible
cache_stat = os.stat(list_cache)
age = time.time() - cache_stat.st_mtime
# st_time can have a greater precision than time, removing
# float precision makes sure age will never be a negative
# number.
age = int(time.time()) - int(cache_stat.st_mtime)
else:
# if filelist does not exists yet, mark it as expired
age = opts.get('fileserver_list_cache_time', 20) + 1
Expand Down
1 change: 0 additions & 1 deletion salt/modules/win_pkg.py
View file
Open in desktop
Expand Up @@ -699,7 +699,6 @@ def refresh_db(**kwargs):
include_pat='*.sls',
exclude_pat=r'E@\/\..*?\/' # Exclude all hidden directories (.git)
)

return genrepo(saltenv=saltenv, verbose=verbose, failhard=failhard)


Expand Down
17 changes: 11 additions & 6 deletions tests/integration/modules/test_win_pkg.py
View file
Open in desktop
Expand Up @@ -15,7 +15,8 @@
import salt.utils.files
import salt.utils.platform

CURL = os.path.join(RUNTIME_VARS.FILES, 'file', 'base', 'win', 'repo-ng', 'curl.sls')
REPO_DIR = os.path.join(RUNTIME_VARS.FILES, 'file', 'base', 'win', 'repo-ng')
CURL = os.path.join(REPO_DIR, 'curl.sls')


@skipIf(not salt.utils.platform.is_windows(), 'windows test only')
Expand All @@ -33,8 +34,10 @@ def test_adding_removing_pkg_sls(self):
Test add and removing a new pkg sls
in the windows software repository
'''
def _check_pkg(pkgs, exists=True):
self.run_function('pkg.refresh_db')
def _check_pkg(pkgs, check_refresh, exists=True):
refresh = self.run_function('pkg.refresh_db')
self.assertEqual(check_refresh, refresh['total'],
msg='total returned {0}. Expected return {1}'.format(refresh['total'], check_refresh))
repo_data = self.run_function('pkg.get_repo_data', timeout=300)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timeout=30 it just opens an existing file and reads it into ram. 30 seconds is heaps

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like that change was made intentionally to also try to stabilize this flaky test so I dont want to remove it quit yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could also call pkg.refresh_db with verbose=True and failhard=False
The data structure return will be this even if it fails (with data not like the example below).

{
    "local": {
        "total": 0,
        "success": 0,
        "success_list": {},
        "failed_list": {},
        "failed": 0
    }
}

With out failhard=False, on bad package definition it will raise an error which returns text and not a data structure like above.

repo_cache = os.path.join(RUNTIME_VARS.TMP, 'rootdir', 'cache', 'files', 'base', 'win', 'repo-ng')
for pkg in pkgs:
Expand All @@ -51,7 +54,7 @@ def _check_pkg(pkgs, exists=True):

pkgs = ['putty', '7zip']
# check putty and 7zip are in cache and repo query
_check_pkg(pkgs)
_check_pkg(pkgs, 2)

# now add new sls
with salt.utils.files.fopen(CURL, 'w') as fp_:
Expand All @@ -74,11 +77,13 @@ def _check_pkg(pkgs, exists=True):
'''))
# now check if curl is also in cache and repo query
pkgs.append('curl')
_check_pkg(pkgs)
for pkg in pkgs:
self.assertIn(pkg + '.sls', os.listdir(REPO_DIR))
_check_pkg(pkgs, 3)

# remove curl sls and check its not in cache and repo query
os.remove(CURL)
_check_pkg(['curl'], exists=False)
_check_pkg(['curl'], 2, exists=False)

def tearDown(self):
if os.path.isfile(CURL):
Expand Down