Read dataset metadata

[Gautzilla]

🐳 WHATISHAPPENING

We're having some trouble when deserializing osekit.public_api.dataset.Dataset instances from .json files that are linked to a lot of audio files, because it implies fully deserializing all the analysis datasets and it can take quite some time (I have to open each file to read the metadata because the AudioFiles are deserialized too on dataset deserialization

🐳 HOWDOWEFIXIT

This is a first (naive) step: now, deserializing the public Dataset will only store the path of the analysis datasets jsons:

Then, these analysis datasets will only be deserialized on request, e.g.:

sds = dataset.get_dataset("maxicoolyo") # Deserializes, stores and returns the SpectroDataset

[coveralls]

coverage: 98.805% (+0.009%) from 98.796%
when pulling 4135adb on Gautzilla:read-dataset-metadata
into 80e88b1 on Project-OSmOSE:main.

[ElodieENSTA]

I tested it in APLOSE and it works!

I just found that calling the path to the json "dataset" is a bit misleading as this is not a dataset we can get with get_dataset

[Gautzilla]

I just found that calling the path to the json "dataset" is a bit misleading as this is not a dataset we can get with get_dataset

Yup, I agree that the distinction between the public dataset and the core/analysis datasets really is unclear...

Maybe I should take a dive in the code and rename everything more explicitly? e.g. systematically rename dataset with analysis_dataset when it concerns an analysis (core) dataset, and leave the raw dataset for everything that concerns the Public API dataset?

@mathieudpnt might have an opinion on this too

[mathieudpnt]

I just found that calling the path to the json "dataset" is a bit misleading as this is not a dataset we can get with get_dataset

Yup, I agree that the distinction between the public dataset and the core/analysis datasets really is unclear...

Maybe I should take a dive in the code and rename everything more explicitly? e.g. systematically rename dataset with analysis_dataset when it concerns an analysis (core) dataset, and leave the raw dataset for everything that concerns the Public API dataset?

@mathieudpnt might have an opinion on this too

IMO dataset belongs to public_API, I would call analysis what we currently call "dataset" from core_API and analysis_config what we call currently "analysis" in core_API

[Gautzilla]

I opened #351 to address the renaming issues!

[Gautzilla]

I tested it in APLOSE and it works!

Cool! So do we merge it as it is? Or do you still need anything else?