Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] Aim revamped SDK #2661

Merged
merged 35 commits into from
Apr 27, 2023
Merged

[feat] Aim revamped SDK #2661

merged 35 commits into from
Apr 27, 2023

Conversation

alberttorosyan
Copy link
Member

Implementation of the Aim Python SDK.

Provides clean interfaces for working with the metadata concepts, such as Container, Sequence, Object, etc.

Core components

  • Module aim.sdk.core.interfaces; declaration of the interfaces (pure abstract)
  • Module aim.sdk.core; implementation of Container, Sequence, Object and collections of them based on the aimrocks storage backend and TreeView abstraction
  • Module aim.sdk.core.type_utils; implementation of Aim type sytem utilities
  • Module aim.sdk.core.repo; implementation of a wrapper class for theaim.Repo class, with abstractions for supporting different types of Containers/Sequences.

Web APIs

  • Endpoint api/objects/search; unified API for querying data from Aim (including types non-compatible with the new SDK, such as aim.Run, aim.Metric, aim.Images)
  • Support for query expression and query object type selection.
  • cURL & JS request examples:
curl --location --request GET 'http://127.0.0.1:43800/api/objects/search?type_=Sequence&q=sequence.name=='\''metric'\'''
var data = "";

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
  if(this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("GET", "http://127.0.0.1:43800/api/objects/search?type_=Sequence&q=sequence.name=='metric'");

xhr.send(data);

@alberttorosyan alberttorosyan marked this pull request as ready for review April 27, 2023 00:30
Copy link
Contributor

@mihran113 mihran113 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments/thoughts for discussion later on, overall looks great!

exists = False

if self._container._is_readonly and not exists:
raise ValueError('Cannot create sequence from a readonly container.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should have been a key error?

for name in self._sequence_tree.subtree(ctx_idx).keys():
yield self._sequence_cls(self._container, name=name, context=ctx_idx)

def __getitem__(self, item: Union[str, Tuple[str, Dict]]) -> Sequence:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general does it make sense to do some write operations on a __getitem__ method? Aren't there any alternatives to this design?

break
yield hash_, name, context

def filter(self, expr: str) -> ABCSequenceCollection['Sequence']:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to allow filtering at all on a limited collection?

step = self._info.next_step

if self._info.empty:
self._tree[KeyNames.INFO_PREFIX, 'creation_time'] = utc_timestamp()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add also step/track call counts here as well

if step is None:
step = self._info.next_step

if self._info.empty:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add some logic for per type sequence count to use later for stats

@alberttorosyan
Copy link
Member Author

resolves #2689

@alberttorosyan alberttorosyan merged commit 19d0af6 into main Apr 27, 2023
@alberttorosyan alberttorosyan deleted the feature/aim-python-sdk branch April 27, 2023 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants