Created dataset is not displayed under project_name in clearml dashboard under /PROJECTS

[fadishaar84]

Describe the bug

A clear and concise description of what the bug is.
I have created a new testing dataset using test-datasets as project_name, if we go to clearml dashboard to display the details of this created dataset we found that the project test-datasets has been created under PROJECTS but it is empty. If we try to delete it will get an error saying that "Project has associated non-empty datasets" which means that it contains the dataset we have created but cannot be listed under this "test-datsets" project. If we go to DATASETS tab, we can see the dataset there. So, the issue is related to displaying the dataset detail under project_name. Note that, if we also go to all experiments we can see the created dataset info there. Thank you.

To reproduce

Exact steps to reproduce the bug. Provide example code if possible.
project='test-datasets'
dataset = Dataset.create(
dataset_name=name,
dataset_project=project,
dataset_tags=tags,
parent_datasets=[parent_id] if parent_id else '',
output_uri=MINIO_URL,
description='{}-{}'.format(name, project)
)
dataset.add_files(
path=path_to_dataset,
recursive=True
)
dataset.upload()
dataset.finalize()

Expected behaviour

What is the expected behaviour? What should've happened but didn't?

Environment

• Server type (self hosted)
• ClearML SDK Version
• ClearML Server Version (Only for self hosted). WebApp: 1.12.1-397 • Server: 1.12.1-397 • API: 2.26
• Python Version 3.8.5
• OS (Linux )
  
  
  

Related Discussion

If this continues a slack thread, please provide a link to the original slack thread.

[mrsahabu]

Facing the similar issue

[ainoam]

@fadishaar84 @mrsahabu This is actually the intended behaviour: The Projects page shows experiments and models, while datasets are shown on the Datasets page. In your example - Your newly created project shows that it has no experiments, but warns you additional content exists, and you should archive it in order to delete the project.

When viewing "All experiments", is this with "Show hidden projects" turned on?

[fadishaar84]

@ainoam Thanks for your answer, but I think in this case, we should have the project created inside DATASETS and not inside PROJECTS. Otherwise, it does not make sense to create an empty project under PROJECTS related to the created dataset.

[ainoam]

@fadishaar84 Projects are not created "inside" pages. Different pages show the project structure as relevant to their applicable content (e.g. use the "project view" in the datasets page to see your datasets grouped by projects).

Can you comment on your "hidden projects" configuration?

[fadishaar84]

@fadishaar84 @mrsahabu This is actually the intended behaviour: The Projects page shows experiments and models, while datasets are shown on the Datasets page. In your example - Your newly created project shows that it has no experiments, but warns you additional content exists, and you should archive it in order to delete the project.

When viewing "All experiments", is this with "Show hidden projects" turned on?

no it is with "show hidden projects" turned off.

[pollfly]

Hey @fadishaar84! Just letting you know that the issue of hidden experiments appearing in search results when "show hidden projects" is disabled has been resolved in the recently released clearml-server 1.13.0.