[CLI] race condition with multiple users using wandb on the same machine

[martenlienen]

Description

If a user starts a new project without a wandb directory in the project root, wandb will log to /tmp/debug-cli.log as set in the following lines.

https://github.com/wandb/client/blob/1d623ac04139d5a72c9f7691e18517e4950849b7/wandb/cli/cli.py#L47-L49

In our university environment we have many students using wandb on the same server. The first student creates the temp logging file and all others get an error like the following because of this.

Of course, this can be fixed by mkdir wandb in the project directory, but I still think this should be fixed.

Wandb features
What lines of wandb code are you using? This could be wandb.init(), wandb.log(), or features like the Keras callback, the Hugging Face integration, or PyTorch Lightning integration, for example.

How to reproduce

See description.

Environment

• Ubuntu Server 20.04
• two users using wandb simultaneously for the first time

[vanpelt]

@martenlienen we automatically attempt to call mkdir wandb in the same directory as the script when wandb.init() is called. If that file system isn't writable, we callback to the system TMPDIR. We unfortunately don't want to do that when running cli commands because it might create lots of wandb directories all over your file system. An alternative approach here might be to use a tmp directory in the current users home directory if we can't find a wandb directory that exists.

[vanpelt]

As a temporary workaround, you can set the TMPDIR environment variable to a directory in your home directory.

mkdir ~/tmp
export TMPDIR=~/tmp

[martenlienen]

Or you could add some (semi-)random suffix to the log file path.

[github-actions]

This issue is stale because it has been open 60 days with no activity.

[pedrocolon93]

This is still happening. I am trying the fix for the TMPDIR variable