uvcdat writing thousands of tempfiles to users' .uvcdat directory

[aashish24]

We have been having problems with uvcdat writing thousands of tempfiles to users' .uvcdat directory and maxing out their disk space. For example, one of our users has found 39500 files (totaling 1.5 GB) of the form: %2Fhome%2Flcarrier%2F.uvcdat%2Funtitled.vt_tmp_9999. Anyone know why this would be happening?

— Tom

[aashish24]

@doutriaux1 do you want to look into this? It seems to me that .vt files are vistrails files but I could be wrong.

[doutriaux1]

yes they are vistrails files, I'm not sure what the best way to move forward is here, considering recent development

[aashish24]

@ThomasMaxwell did the user ran the GUI? these files could the provenance files used. If that's the case, we may have to put a workaround for now.

[ThomasMaxwell]

Yes, the user was using the GUI. These could be provenance files but I don’t know why there would be so many of them.

From: Aashish Chaudhary <notifications@github.commailto:notifications@github.com>
Reply-To: UV-CDAT/uvcdat <reply@reply.github.commailto:reply@reply.github.com>
Date: Thursday, June 25, 2015 at 4:27 PM
To: UV-CDAT/uvcdat <uvcdat@noreply.github.commailto:uvcdat@noreply.github.com>
Cc: "Maxwell, Thomas P. (GSFC-606.2)[SCIENCE APPLICATIONS INTL CORP]" <thomas.maxwell@nasa.govmailto:thomas.maxwell@nasa.gov>
Subject: Re: [uvcdat] uvcdat writing thousands of tempfiles to users' .uvcdat directory (#1428)

@ThomasMaxwellhttps://github.com/ThomasMaxwell did the user ran the GUI? these files could the provenance files used. If that's the case, we may have to put a workaround for now.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1428#issuecomment-115387534.

[aashish24]

There could be multiple reasons. Vistrails create temporary file for few things. I will see if I can reproduce.

[doutriaux1]

@chaosphere2112 @rexissimus can one of you take a quick stab at this?

[rexissimus]

These are autosave files that are created every 2 minutes. They are deleted when closing uv-cdat. But it could lead to problems when leaving uv-cdat running for long periods of time. In this case it looks like uv-cdat has been running for 2 months?

We could fix this by limiting the number of temporary files somehow, but I am not sure this is worth looking into at this point?

[chaosphere2112]

@rexissimus We're shipping a release of the existing GUI, so, yeah, it'd be nice to clean this up. Can you just shove them into /tmp instead of .uvcdat?

[rexissimus]

Then you would probably lose the autosave if the machine reboots and /tmp is wiped. I will look into limiting the number of temporary files instead.

[doutriaux1]

why do we need a new file every two minutes? Can't the new file contain the necessary informations from the previous one and delete the old one?

[rexissimus]

Having 2 autosaves may be useful if there is a crash while writing it. Having more could possibly be useful if the loaded file gets corrupted while in memory and you need to restore from some time before. But this is unlikely to happen. I added a fix that limits the files to between 30-60: CDAT/VisTrails#55