Need better handling of temporary files

[mapserver-bot]

Reporter: dmorissette
Date: 2010/03/03 - 19:50
Trac URL: http://trac.osgeo.org/mapserver/ticket/3354
At the moment, we write some temporary files in the web-accessible IMAGEPATH directory, this was a poor practice but still okay for some uses in the past (such as writing CONNECTIONTYPE WMS and WFS responses while we process them), but as our need for temporary files increase, we need to ensure that temp files are handled in a proper and safer way.

One option could be to adopt the use of a $MS_TMPDIR environment variable to specify the temporary directory path. If $MS_TMPDIR is not set, then we would fallback on $TMPDIR (commonly used on Unix), and if not, on /tmp on Unix/Linux, or some path TBD on Windows.

We could also look into the use of system calls to more safely handle temp files. One possibility is tmpfile(), but the drawback of this one is that it automatically deletes the files once it's closed and in some cases we don't want that.

[mapserver-bot]

Author: assefa
Date: 2010/03/03 - 20:24
On windows, environment variables TEMP and TMP could be used for testing, and then c:\temp as last resort.

[mapserver-bot]

Author: dmorissette
Date: 2010/03/03 - 20:26
See also #3355 (about OGC-specific uses of temp files)

[mapserver-bot]

Author: sdlime
Date: 2010/03/03 - 21:32
Agreed this is a need. Very few use cases need to use the image path, that is, only for writing images. What about explicitly setting a path in a mapfile (e.g. TEMPPATH) and then falling back to env vars?

Steve

[mapserver-bot]

Author: aboudreault
Date: 2010/06/09 - 15:17
I agree with the TMPPATH in the mapfile. If this var is now set, on Linux, then check $MS_TMPDIR env var, $TMPDIR, otherwise use /tmp. On Windows, TMPPATH, $MS_TMPDIR, $TEMP, $TMP, otherwise use c:\temp.

Does everyone agree with this behavior? What about MAC ?

[mapserver-bot]

Author: aboudreault
Date: 2011/01/21 - 20:42
I've just committed the RFC 66 in 9109365 (r10883). Assefa will check and fix the Windows part as soon as he can.

[mapserver-bot]

Author: assefa
Date: 2011/01/21 - 22:17
Alan,

builds on windows with dafc9c8 (r10888).
Order of things on windows is as follows.

1 - TMPATH
2 - MS_TMPPATH env variable
3 - The path specified by the TMP environment variable.
4 - The path specified by the TEMP environment variable.
5 - The path specified by the USERPROFILE environment variable.
6 - The Windows directory.

I have tested this with the sld temporary file.

[mapserver-bot]

Author: assefa
Date: 2011/01/22 - 19:45
Alan,

I would like to be able to use the msTmpFile to get a temporary full path file name for a given filename. Right now the function allows the possibility to pass path and extension but not a file name.
In my case I would like to create a file and I know the name I want but I do not have to worry about the temporary directory location.
Does this make sense? We could add an additional argument to the function.

[mapserver-bot]

Author: assefa
Date: 2011/01/23 - 17:09
Forgot to note that we can also have msTmpPath that will return the temp path and I can construct the full path. Both ways are the same to me.

[mapserver-bot]

Author: aboudreault
Date: 2011/01/24 - 16:10
I assume you wil always delete that file after the task... otherwise it wouldn't safe to have a static filename. I can effectively separate the msTmpFile function and create a msTmpPath.

[mapserver-bot]

Author: aboudreault
Date: 2011/01/24 - 16:44
Committed the ''msTmpPath'' function in ae26466 (r10900). The function will return the temporary path based on the os and arguments. ie: (linux)

msTmpPath(NULL,NULL,NULL); // returns /tmp
msTmpPath(NULL,"/a_map_path",NULL); // returns /a_map_path/tmp
msTmpPath(map,NULL,NULL); // with temppath set, returns /the_temppath/
msTmpPath(NULL,NULL,"/specific_path/tmp/"); // returns /specific_path/tmp/
// etc.

Please confirm that it fixes your need.

[mapserver-bot]

Author: assefa
Date: 2011/01/24 - 21:40
Thanks Alan for adding that.
I use it in my case for #3662. I do not delete the tmp file and use it if it exists, the idea being able to use the tmp file as a cache for a symbol file downloaded.

[mapserver-bot]

Author: aboudreault
Date: 2011/02/01 - 20:29
Closing the ticket.

[mapserver-bot]

Author: havatv
Date: 2011/12/03 - 10:30
As far as I can see, this has not been documented. Reopening and moving to documentation. #4103 is related.

[mapserver-bot]

Author: havatv
Date: 2011/12/03 - 12:24
Documentation of TEMPPATH and MS_TEMPPATH was added to the mapfile web document for 6.0 and trunk in MapServer/MapServer-documentation@9d2849e (r12839). What else should be documented?

[mapserver-bot]

Author: aboudreault
Date: 2011/12/03 - 15:13
Looks good to me. Thanks a lot havatv!