Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle different dataset types in a better way #963

Closed
antonymilne opened this issue Jul 6, 2022 · 3 comments
Closed

Handle different dataset types in a better way #963

antonymilne opened this issue Jul 6, 2022 · 3 comments
Labels
Python Pull requests that update Python code

Comments

@antonymilne
Copy link
Contributor

The code that handles different datasets types uses the full to the dataset class, like this:

def is_metric_node(self):
"""Check if the current node is a metrics node."""
return (
self.dataset_type
== "kedro.extras.datasets.tracking.metrics_dataset.MetricsDataSet"
)

This is quite verbose, awkward and also fragile, because if the definition moves in kedro (which it will do when we move to kedro-dataset). Let's come up with a better system for it, e.g. just using type(dataset).__qualname__ without including the module path.

Possible gotchas:

  • kedro-viz needs to work without e.g. the underlying dependencies for e.g. kedro.extras.datasets.matplotlib.matplotlib_writer.MatplotlibWriter. This might be why it's done the way it is currently (string matching) rather than using isinstance
  • need to think about what to do if __qualname__ is not unique, e.g. in the case of json.JSONDataSet vs. plotly.JSONDataSet
@rashidakanchwala
Copy link
Contributor

I agree, json.JSONDataset, tracking.JSONDataSet, and plotly.JSONDataSet -- can we have more unique qualname_

@rashidakanchwala
Copy link
Contributor

This is solved now. so we can close it :)

@tynandebold
Copy link
Member

Improved via #1214.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Python Pull requests that update Python code
Projects
Status: Done
Development

No branches or pull requests

3 participants