Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Proxystore v0.5.0 #101

Merged
merged 2 commits into from
May 14, 2023
Merged

Update to Proxystore v0.5.0 #101

merged 2 commits into from
May 14, 2023

Conversation

gpauloski
Copy link
Collaborator

@gpauloski gpauloski commented May 13, 2023

This PR updates Colmena to use ProxyStore v0.5.0. The synthetic-data demo app is also updated.

Here's the changes to colmena/ (i.e., ignoring the demo app and doc changes):

  1. The Store now has .config() and .from_config() methods. This greatly simplifies the storing the ProxyStore configuration within each Result object and recreating the Store if necessary when serializing the Result.
  2. The tests which use ProxyStore needed to be updated to use the new Connector model (a straightforward change).
  3. The format of data returned by store_proxy_stats (which is stored in Result) has changed. While the data is more informative/human-readable, this is a breaking change for any code that parses Result.proxy_timing. An example can be found here, and a sample entry in Result.proxy_timing is provided below.
>>> result.proxy_timing
{
    "FileKey(filename='7f4e12ba-622f-4946-ac01-9e3e5fee879d')": {
        "attributes": {"store.get.object_size": 1000165},
        "counters": {"store.get.cache_misses": 1},
        "times": {
            "store.get.connector": {
                "count": 1,
                "avg_time_ms": 189.037,
                "min_time_ms": 189.037,
                "max_time_ms": 189.037,
                "last_time_ms": 189.037,
                "last_timestamp": 1683940464.067098,
            },
            "store.get.deserialize": {
                "count": 1,
                "avg_time_ms": 427.167,
                "min_time_ms": 427.167,
                "max_time_ms": 427.167,
                "last_time_ms": 427.167,
                "last_timestamp": 1683940464.0675457,
            },
            "store.get": {
                "count": 1,
                "avg_time_ms": 689.462,
                "min_time_ms": 689.462,
                "max_time_ms": 689.462,
                "last_time_ms": 689.462,
                "last_timestamp": 1683940464.0675554,
            },
            "factory.resolve": {
                "count": 1,
                "avg_time_ms": 708.168,
                "min_time_ms": 708.168,
                "max_time_ms": 708.168,
                "last_time_ms": 708.168,
                "last_timestamp": 1683940464.0675676,
            },
            "factory.call": {
                "count": 1,
                "avg_time_ms": 768.181,
                "min_time_ms": 768.181,
                "max_time_ms": 768.181,
                "last_time_ms": 768.181,
                "last_timestamp": 1683940464.067604,
            },
        },
    },
}

@gpauloski gpauloski requested a review from WardLT May 13, 2023 01:36
Copy link
Collaborator

@WardLT WardLT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM minus a suggested edit in the docs. Thanks, @gpauloski !

docs/quickstart.rst Outdated Show resolved Hide resolved
@gpauloski gpauloski merged commit f0e2464 into master May 14, 2023
1 check passed
@gpauloski gpauloski deleted the proxystore-v0.5.0 branch May 14, 2023 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants