Can't download dataset metadata without downloading the dataset itself

[joaquinvanschoren]

Description

There are situations where one only wants to see the meta-data of a given dataset. The current get_dataset always downloads and parses the dataset itself. This is slow(!), and can cause errors if the files are really large.

It would be nice to have an option (or another call) to download the meta-data only.

To make matters worse, the arff parser requires many times the file size for the download and parsing. E.g. downloading a 2GB dataset requires more than 16GB disk space. This fails in colab notebooks: they crash because of disk space limits. I would download the csv version directly, but this requires the file url or file_id, neither of which are returned by list_dataset and thus have to be retrieved with get_dataset. So, there is currently no way to download larger OpenML dataset into colab notebooks without manually looking up the file_ids.

Steps/Code to Reproduce

Download any large dataset (e.g. SVHN):

openml.datasets.get_dataset(41081)

Expected Results

An OpenMLDataset object without the actual dataset, or a dict with the meta-data.

Actual Results

get_dataset is very slow, or crashes if disk space is not sufficient.

Versions

Darwin-18.2.0-x86_64-i386-64bit
Python 3.6.4 |Anaconda custom (64-bit)| (default, Jan 16 2018, 12:04:33)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
NumPy 1.15.3
SciPy 1.1.0
Scikit-Learn 0.20.0
OpenML 0.7.0

[janvanrijn]

Agreed. Similar for tasks, where the split file is commonly downloaded (potentially 10 times the size of the dataset)

[mfeurer]

Agreed, being able to disable this would be a great addition. I think actually be a better solution than the one proposed in #346, what do you think @amueller ?

[PGijsbers]

I agree that metadata should be accessible easily without downloading the whole dataset. I think adding a parameter to the get_dataset method makes sense, e.g.:

def get_dataset(dataset_id: int, download_data: bool=False) -> OpenMLDataset:
    """ Download the OpenML dataset representation, optionally also download actual data file. 

    This function is thread/multiprocessing safe.
    This function uses caching. A check will be performed to determine if the information has
    previously been downloaded, and if so be loaded from disk instead of retrieved from the server.

    Parameters
    ----------
    dataset_id : int
        Dataset ID of the dataset to download
    download_data : bool
        If True, also download the data file. Beware that some datasets are large and it might make
        the operation noticeably slower. Metadata is also still retrieved.
        If False, create the OpenMLDataset and only populate it with the metadata.
        The data may later be retrieved through the `OpenMLDataset.get_data` method.

    Returns
    -------
    dataset : :class:`openml.OpenMLDataset`
        The downloaded dataset."""

We would then also have to update the get_data method to check if the data is already present in cache (and download it if not). Edit: looks like retrieve_class_labels will have issues too, we'll have to see how to work around that (if it's an issue).

I think not downloading the data by default is fine behavior. Right now to look at the data, you always call the get_data method anyway.

For those looking for a solution right now to download qualities metadata without downloading the actual data, use this code-snippet (tested with OpenML 0.7.0):

from openml.datasets.functions import _get_dataset_qualities, DATASETS_CACHE_DIR_NAME
from openml.datasets.dataset import _check_qualities
from openml.utils import _create_cache_directory_for_id

def get_qualities(dataset_id):
   did_cache_dir = _create_cache_directory_for_id(DATASETS_CACHE_DIR_NAME, dataset_id)
   qualities_xml_dict = _get_dataset_qualities(did_cache_dir, dataset_id)
   return _check_qualities(qualities_xml_dict)

[mfeurer]

Functionality added in #644.