Skip to content

Conversation

@Nathanjp91
Copy link
Contributor

@Nathanjp91 Nathanjp91 commented Oct 12, 2023

Problem

Dogfooding revealed ways we'd like to use team and dataset objects regarding deletion and creation

Solution

Changes include

  • Team object now has a way to create a dataset via team.create_dataset(slug)
  • Team object now has a delete_dataset method
  • Dataset now has a delete method, allowing interaction with queries Team.dataset.where(...)[0].delete()
  • Changes to handling of exceptions
  • Ruff formatting changes
  • Black formatter unification

Changelog

Team and Dataset methods now include new ways to create/delete

@linear
Copy link

linear bot commented Oct 12, 2023

IO-1458 Remove DatasetMeta get_dataset_by_id method

DatasetMeta has a method called get_dataset_by_id, but the DatasetMeta object will be a dataset by state, so this should be removed.

IO-1454 Alter behaviour of `DatasetMeta` delete methods

Change behaviour of DatasetMeta object to:

  • Allow dataset.delete() on an instantiated dataset, retrieved from a query.
  • Remove existing dataset_delete method, as it is defunct.

Acceptance criteria

  • A retrieve dataset can be deleted by calling .delete() method on it.

IO-1455 Add dataset creation methods to TeamMeta object

Add a method to TeamMeta that can create a dataset by slug.

ex.

new_dataset = team.create_dataset('a_dataset') # should create a dataset with slug 'a_dataset'

…where team is a TeamMeta instantiated object.

Acceptance Criteria

  • Calling team.create_dataset with a valid slug string, will create, and return, the dataset object.

IO-1456 Add delete_dataset method to TeamMeta object

Add a method to TeamMeta that can delete a dataset by either slug or ID.

ex.

team.delete_dataset(slug='some_dataset')
# --or--
team.delete_dataset(id=1337)

Consider possibility that method could test parameter for string/int type, and delete by slug/ID accordingly.

Acceptance criteria

The following commands will delete a dataset:

  • team.delete_dataset(slug='some_dataset')
  • team.delete_dataset(id=1337)

…if the dataset of that slug or ID exists.

@Nathanjp91 Nathanjp91 requested a review from owencjones October 12, 2023 14:16
Copy link
Contributor

@owencjones owencjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, LGTM. A few comments, but getting the huge black transform out of the way was a good call.

def validate(cls, v: str) -> "TeamSlug":
assert len(v) < cls.max_length, f"maximum length for team slug is {cls.max_length}"
assert len(v) > cls.min_length, f"minimum length for team slug is {cls.min_length}"
assert (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess these ones are where it feels a bit more oppressive to use col88 widths. I might adopt an approach of #fmt: off and do my own formatting to make them look nicer in future.

assert_is(isinstance(dataset_id, int), "dataset_id must be an integer")

dataset_deleted = remove_dataset(client, dataset_id)
dataset_deleted = remove_dataset(self.client, self.id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice little win here.

upload_data(
self._element.name, files, files_to_exclude, fps, path, frames, extract_views, preserve_folders, verbose # type: ignore
)
upload_data(self._element.name, files, files_to_exclude, fps, path, frames, extract_views, preserve_folders, verbose) # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one confuses me. I've not counted, but that can't be less than 88 chars can it?

@property
def item_ids(self) -> List[UUID]:
"""_summary_
"""Item ids attached to the stage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually quite surprised Sphinx didn't moan about this (pre-this change) but maybe we have it excluded in sphinx steps atm

A tuple containing a list of exceptions and the number of datasets deleted
"""
exceptions = []
dataset_deleted = -1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very C 🤣

upload_data(
self.datasets[0].name, files, files_to_exclude, fps, path, frames, extract_views, preserve_folders, verbose # type: ignore
)
upload_data(self.datasets[0].name, files, files_to_exclude, fps, path, frames, extract_views, preserve_folders, verbose) # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again I actually think I disagree with black on this one, but not so strongly I would write a plugin, which is a decent bar to have actually!

assert str(query_string_2) == "?foo=bar&baz=qux"

query_string_3 = QueryString(dict())
query_string_3 = QueryString({})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another of the ones that irks me a bit of the suggested changes - because is {} an empty dict or set.

I guess overall it doesn't matter in this case, but if you were trying to instantiate a set, it had better allow = set()

@@ -1,6 +1,5 @@
from pathlib import Path

import pytest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This being unused means this file is missing a run block at the bottom, so it'll only run in set. Not the end of the world, but worthy of note.


def test_delete_dataset_calls_delete_by_id_as_appropriate(
base_config: DarwinConfig, _delete_by_id_mock: Mock, _delete_by_slug_mock: Mock
@mark.parametrize(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

@Nathanjp91 Nathanjp91 merged commit e958307 into master Oct 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants