Skip to content

Conversation

@mercuryseries
Copy link
Contributor

@mercuryseries mercuryseries commented Feb 24, 2025

Changelogs

  • New Features:

    • Added list_models utility function.
    • Introduced a "Create a Model" tutorial.
    • Added model_artifact_id as a computed property in ResultsMetadataV2.
  • Improvements:

    • Added code_url as an alias for github_url in ResultsMetadataV1.
    • Added report_url as an alias for paper_url in ResultsMetadataV1.
  • Bug Fixes:

    • Fixed a bug in result evaluation by using a shallow copy of subsets instead of a deep copy.
    • Fixed a bug in the get_model method (wrong endpoint URL).
    • Fixed a bug in the _get_v2_benchmark method (TypeError: got multiple values for keyword argument 'split')
    • Fixed typings in the upload_results method for results param.
    • Corrected small typos in docstrings.

Checklist:

  • Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.
  • Add tests to cover the fixed bug(s) or the newly introduced feature(s) (if appropriate).
  • Update the API documentation if a new function is added, or an existing one is deleted.
  • Write concise and explanatory changelogs above.
  • If possible, assign one of the following labels to the PR: feature, fix, chore, documentation or test (or ask a maintainer to do it for you).

Demo Videos

Model Landing and Detail Pages / Leaderboard
0_landing_detail_pages.mp4
Create a Model
2_create_model.mp4
Attach a Model to results
3_attach_model_to_results.mp4
List models/Load a specific Model
1_list_models_load_model.mp4

@mercuryseries mercuryseries added feature Annotates any PR that adds new features; Used in the release process fix Annotates any PR that fixes bugs labels Feb 27, 2025
@mercuryseries mercuryseries marked this pull request as ready for review February 27, 2025 16:07
@mercuryseries mercuryseries changed the title Model artifact remaining changes Client: Model artifact remaining changes Feb 27, 2025
@mercuryseries mercuryseries self-assigned this Feb 27, 2025
… of a deepcopy

Co-authored-by: Cas Wognum <cwognum@users.noreply.github.com>
Co-authored-by: Julien St-Laurent <jstlaurent@users.noreply.github.com>
Copy link
Contributor

@jstlaurent jstlaurent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@cwognum cwognum requested a review from Copilot February 28, 2025 14:17
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@cwognum cwognum requested a review from Copilot February 28, 2025 14:35
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This PR introduces several enhancements to the Polaris Hub client and related modules. Key changes include new utility methods for listing models, adjustments to benchmark and dataset formatting (now using artifact IDs/slugs), and updates to model upload endpoints.

Reviewed Changes

File Description
polaris/hub/client.py Updated imports and renamed endpoints; added list_models method and revised benchmarks/datasets returning artifact IDs.
polaris/evaluate/_metadata.py Added computed field for model_artifact_id and changed field aliases.
polaris/dataset/_subset.py Replaced a deep copy with a shallow copy for subset copying.
docs/tutorials/create_a_model.ipynb Introduced a tutorial for creating and uploading a model.
docs/tutorials/create_a_dataset.ipynb Corrected a typo in unit formatting.

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

polaris/hub/client.py:420

  • [nitpick] The docstring now refers to the benchmark as 'owner/benchmark_slug', but the code uses 'artifactId'. Please confirm that artifactId reliably represents the benchmark slug.
v1_benchmarks = [benchmark["artifactId"] for benchmark in v1_data]

polaris/hub/client.py:949

  • The upload_model method now uses a PUT request to update the model at a URL containing model.artifact_id instead of a POST to create a new model. Please ensure that this change in HTTP method and endpoint is intended and covered by tests.
url = f"/v2/model/{model.artifact_id}"

Copy link
Collaborator

@cwognum cwognum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @mercuryseries !

@roselynh100 roselynh100 merged commit c4079da into main Apr 17, 2025
19 checks passed
@jstlaurent jstlaurent deleted the model-artifact-remaining-changes branch May 16, 2025 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Annotates any PR that adds new features; Used in the release process fix Annotates any PR that fixes bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants