Cannot create .tmp in Google Colab

[fonnesbeck]

Trying to fit NODE models on Colab results in an error due to the inability to create .tmp in the current directory:

OSError                                   Traceback (most recent call last)
<ipython-input-24-184a686ccb34> in <module>()
     45     model_config=model_config,
     46     optimizer_config=optimizer_config,
---> 47     trainer_config=trainer_config,
     48 )

2 frames
/usr/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
    221             return
    222     try:
--> 223         mkdir(name, mode)
    224     except OSError:
    225         # Cannot rely on checking for EEXIST, since the operating system

OSError: [Errno 95] Operation not supported: '.tmp'

There is not an obvious place to pass an alternative location.

Running pytorch_tabular 0.6.0 and pytorch 1.9 on Colab Pro.

[rajshah4]

I was able to overcome this issue when running notebooks in colab by changing the working directory to tmp prior to defining the configs by using this command: %cd tmp in colab

[manujosephv]

This is because of a slightly hacky workaround I used to maintain the versions in experiment tracking. If you run multiple experiments with same names, I use a counter stored in the disk to add the version to the name. And for this, the code saves a file in .tmp . Maybe the issue here is the lack of access to create a .tmp or may be there is already another .tmp in colab. If it's the latter, I can change the folder name to be more specific to Pytorch Tabular - something like .pt_tmp

[manujosephv]

Have pushed a new version and have bundled this fix. Now by default it tries to save the file in .pt_tmp and if needed it can be changed with a parameter in ExperimentConfig

[rajshah4]

Hmm, no that isn't working still. I get this similar error on the Basic Usage notebook in Colab. I can see the the .pt_tmp file. It is created.
But for some reason when the fit gets called, it is having this error:

<ipython-input-9-ccd69e8e5e51> in <module>()
----> 1 tabular_model.fit(train=train, validation=val)

13 frames
/usr/lib/python3.7/os.py in makedirs(name, mode, exist_ok)
    221             return
    222     try:
--> 223         mkdir(name, mode)
    224     except OSError:
    225         # Cannot rely on checking for EEXIST, since the operating system

FileNotFoundError: [Errno 2] No such file or directory: '' 

Is there something during fit, which colab doesn't like at the root? (Other files that are being created?) If I do a change directory to say tmp, then the fit will run fine.

[fonnesbeck]

Yes, this fails for me as well. There is no tmp to cd to, and I am not allowed to create .pt_tmp.

[manujosephv]

Are you guys, by any chance, using one of the tutorial notebooks as a starting point? Because I just noticed something. There is a os.chdir in the beginning.. (which should have been removed)

If we do not do the chdir, this .tmp problem is not an issue. At least for me.

[rajshah4]

Yes, i was using the tutorial notebooks for testing - removing os.chdir gets everything working on Colab for me. (Both Notebooks 1 and 2)

[manujosephv]

have commented out the chdir from the tutorials and updated the main branch..