Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove outdated items from tool cache when reloading the toolbox #3729

Merged
merged 3 commits into from Mar 17, 2017

Conversation

Projects
None yet
3 participants
@mvdbeek
Copy link
Member

commented Mar 8, 2017

This should prevent some slightly weird things from happening when running galaxy with multiple web processes, like repository updates without a version bump that appear to not get updated (until a galaxy restart).
I wouldn't bet on it, but I think it may also fix the first issue in http://dev.list.galaxyproject.org/tool-problems-data-manager-fetch-ncbi-taxonomy-amp-ggplot2-td4670612.html .

@mvdbeek mvdbeek added this to the 17.05 milestone Mar 8, 2017

@mvdbeek mvdbeek force-pushed the mvdbeek:cleanup_tool_cache branch from 5536a34 to 7bf6a58 Mar 9, 2017

@@ -22,6 +36,8 @@ def expire_tool(self, tool_id):
del self._tools_by_path[config_filename]

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Mar 9, 2017

Member
             del self._hash_by_tool_paths[config_filename]

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Mar 9, 2017

Author Member

good catch!

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Mar 9, 2017

Member

Wrong copy/paste @mvdbeek , s/path/config_filename/ , should fix Travis.

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Mar 9, 2017

Author Member

Oh right, I will just move everything to use config_filename.

def cleanup(self):
"""Remove uninstalled tools from tool cache if they are not on disk anymore or if their content has changed."""
clean_paths = {path: tool.all_ids for path, tool in self._tools_by_path.items() if not os.path.exists(path) or md5_hash_file(path) != self._hash_by_tool_paths[path]}
for path, tool_ids in clean_paths.items():

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Mar 9, 2017

Member

I think s/clean_paths/paths_to_cleanup/ in the previous 2 lines would be clearer.

@mvdbeek mvdbeek force-pushed the mvdbeek:cleanup_tool_cache branch from 53281fa to f30ab0a Mar 9, 2017

Remove hash when expiring tool from tool cache
and rename `clean_paths` to `paths_to_cleanup` (thx @nsoranzo).

@mvdbeek mvdbeek force-pushed the mvdbeek:cleanup_tool_cache branch from f30ab0a to 0d80b88 Mar 9, 2017

@mvdbeek

This comment has been minimized.

Copy link
Member Author

commented Mar 9, 2017

@galaxybot test this

@nsoranzo

This comment has been minimized.

Copy link
Member

commented Mar 9, 2017

@mvdbeek I discovered today that you can restart a single Jenkins build as in the following screenshot and the GitHub "Details" link will be updated automatically as for Travis:

screenshot from 2017-03-09 15 42 25

@dannon

This comment has been minimized.

Copy link
Member

commented Mar 17, 2017

Thanks @mvdbeek!

@dannon dannon merged commit d50ca25 into galaxyproject:dev Mar 17, 2017

5 checks passed

api test Build finished. 263 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 140 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 24 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 580 tests run, 0 skipped, 0 failed.
Details

@mvdbeek mvdbeek deleted the mvdbeek:cleanup_tool_cache branch Jun 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.