Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: Enable Shared Cache Support #511

Draft
wants to merge 6 commits into
base: develop
Choose a base branch
from
Draft

WIP: Enable Shared Cache Support #511

wants to merge 6 commits into from

Conversation

mcfadden8
Copy link
Collaborator

@mcfadden8 mcfadden8 commented Nov 2, 2022 •
edited
Loading

Shared Cache Constraints

  • Store must be configured by user to GLOBAL
  • User must run with SINGLE redundancy scheme

Todo Items

  • Identify files impacted by in a shared cache (GLOBAL store)
  • Document impact for shared files in a shared cache (GLOBAL store)
  • Modify documentation files to describe how global cache works (update old documentation)
  • Remove previous global storage view implementation in preparation for supporting a generalized global cache implementation.

Notable Changes:

  • Updated cmake minimum to 3.14.5 to stop cmake warnings with soon to be deprecated 2.8 support

@mcfadden8 mcfadden8 linked an issue Nov 2, 2022 that may be closed by this pull request
shared-cache - enable shared-cache support in SCR #481
Open
@adammoody
Copy link
Contributor

Let's toss the SCR_CONFIG_KEY_VIEW constant, as well. I think this is only used for storedesc->view.

@mcfadden8
Copy link
Collaborator Author

nice catch!

@adammoody
Copy link
Contributor

Just for reference, we have this description about shared file access now:

https://scr.readthedocs.io/en/v3.0/users/config.html?highlight=global#file-per-process-vs-shared-access

There could be other references scattered around.

Anyway, once this is done and we know how it works, we can come back to update the docs with usage. Just recording placeholders here for now.

@mcfadden8
Copy link
Collaborator Author

mcfadden8 commented Nov 4, 2022 via email

@mcfadden8 mcfadden8 mentioned this pull request Jan 20, 2023
Closed
mcfadden8
mcfadden8 commented Jan 20, 2023
View reviewed changes
src/scr_copy.c
@@ -586,16 +585,16 @@ int main (int argc, char *argv[])
/* get the MPI rank of the file */
value = strndup(entryname + pmatch[1].rm_so, (size_t)(pmatch[1].rm_eo - pmatch[1].rm_so));
if (value != NULL) {
rank = atoi(value);
int rank = atoi(value);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adammoody - clang14 produced some warnings that I fixed. Most were small (like security warnings), but others like this are indicative of a potental bug. I rearranged this slightly so that rank would only be used when it was set. I moved the if block around to what I believe should hopefully be an equivalent of what the inention was previously.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, good catch!

@gonsie gonsie added the WIP label Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adammoody adammoody adammoody approved these changes

Assignees
No one assigned
Labels
WIP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

shared-cache - enable shared-cache support in SCR
3 participants
@mcfadden8 @adammoody @gonsie