Dangerous advice: it looks that by default the dataset is at /data/milatmp1

[dmitriy-serdyuk]

If this is true, Fred is not going to be happy when a lot of people will be hammering NFS.

Given that an experiment is run by a magic script, it is simple to copy data to /Tmp.

[rdevon]

Wait, where is milatmp1 defaulting?

[dmitriy-serdyuk]

In the readme:

torchvision_data_path: /data/milatmp1/hjelmdev/data/

[rdevon]

Ah, right. So one function I need to implement is to designate a local path to which the data is copied before training. Haven't gotten around to it, but I can add it to the todos

[nouiz]

If you have a quick implementation, then good. Otherwise, we are planning to have the local computer cache the networks file system. All new computer we plan to buy will have the hardware for this. Currently, there is a flag to our slurm config to go on computer configured like this.