New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recreation of /tmp/tomcat* directory incase its deleted during file upload #9616

Open
shehzadgee opened this Issue Jun 27, 2017 · 12 comments

Comments

Projects
None yet
10 participants
@shehzadgee

shehzadgee commented Jun 27, 2017

Steps to Reproduce
When spring boot application starts with default value of server.tomcat.basedir in application.properties. It creates two folders in /tmp folder. /tmp/tomcat.xxxxx/.. and /tmp-docbase.xxxx/

These directories are used to save temp files during multipart upload among other needed functions. In production system such as Centos, by default system is configured to delete all /tmp files if not touched for 10 days. Once this file is deleted logs will show this error and upload will fail with following exception

org.springframework.web.multipart.MultipartException: Could not parse multipart servlet request; nested exception is java.io.IOException: The temporary upload location [**/tmp/tomcat.1220970741172837513.8080/work/Tomcat/localhost/ROOT]** is not valid
	at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:111)

Possible Solution:
May be lets check if required temp directory doesn't exist recreate during multipart upload. And upload will work without any additional settings.
Other fixes/suggestions are welcome as well.

Affected Versions
Logs are from Spring Boot v1.4.3.RELEASE but its still same in 1.5.3.RELEASE

Workarounds

  • On Centos 7 update tmp.conf in following three directories /etc/tmpfiles.d/, /run/tmpfiles.d/ and /usr/lib/tmpfiles.d/. with following line
    x /tmp/tomcat*

OR handle programmatically

  • Catch exception in GlobalException Handler and recreate directory and inform user to try upload again.

full_log.txt

@philwebb

This comment has been minimized.

Show comment
Hide comment
@philwebb

philwebb Jun 28, 2017

Member

There's quite a bit of discussion on this in #5009. The documentation for 1.5.0 was also updated with our suggested solution.

Member

philwebb commented Jun 28, 2017

There's quite a bit of discussion on this in #5009. The documentation for 1.5.0 was also updated with our suggested solution.

@kappmeier

This comment has been minimized.

Show comment
Hide comment
@kappmeier

kappmeier Aug 11, 2017

@shehzadgee I know this and the other tickets have been closed, however without a working solution. Can you please elaborate a bit how I would implement your suggested solution of catching the exception and recreating the directory?

Generally, I do not think that something simple as storing a file in a temporary directory during upload should require manual action by the developer or server admin of any kind, so if you reconsider fixing the issue would be great. As for the discussion in #5009 I also think /tmp is the correct directory.

kappmeier commented Aug 11, 2017

@shehzadgee I know this and the other tickets have been closed, however without a working solution. Can you please elaborate a bit how I would implement your suggested solution of catching the exception and recreating the directory?

Generally, I do not think that something simple as storing a file in a temporary directory during upload should require manual action by the developer or server admin of any kind, so if you reconsider fixing the issue would be great. As for the discussion in #5009 I also think /tmp is the correct directory.

@timja

This comment has been minimized.

Show comment
Hide comment
@timja

timja Aug 14, 2017

@philwebb this doesn't seem to be a duplicate can it be reopened?
the other issues are about using another location and the danger of using /tmp

This one is about recreating the files if it can be detected that they have been deleted.

timja commented Aug 14, 2017

@philwebb this doesn't seem to be a duplicate can it be reopened?
the other issues are about using another location and the danger of using /tmp

This one is about recreating the files if it can be detected that they have been deleted.

@shehzadgee

This comment has been minimized.

Show comment
Hide comment
@shehzadgee

shehzadgee Aug 15, 2017

Hi Kappmeier,

I didn't get chance to look into spring boot code to find where this multipart exception is throw. But I handled it my GlobalExceptionHandler in following way. So when I detect that an upload failed because directory is deleted by system cleanup process. I just recreate it and ask user to try upload file again.

So in my humble opinion same check can be placed inside Spring Boot so if directory doesn't exist, it should try to recreate. If it failed due to access rights only than upload should fail.

public  ResponseEntity<?> handle(org.springframework.web.multipart.MultipartException exception) {
		Log.error("handle->MultipartException" + exception.getMessage(),exception);
		// general exception
		if (exception.getCause() instanceof IOException && exception.getCause().getMessage().startsWith("The temporary upload location"))
		{
			String pathToRecreate = exception.getMessage().substring(exception.getMessage().indexOf("[")+1,exception.getMessage().indexOf("]"));
			Set<PosixFilePermission> perms = new HashSet<>();
		    // add permission as rw-r--r-- 644
		    perms.add(PosixFilePermission.OWNER_WRITE);
		    perms.add(PosixFilePermission.OWNER_READ);
		    perms.add(PosixFilePermission.OWNER_EXECUTE);	    
		    perms.add(PosixFilePermission.GROUP_READ);
		    perms.add(PosixFilePermission.GROUP_WRITE);
		    perms.add(PosixFilePermission.GROUP_EXECUTE);
		    FileAttribute<Set<PosixFilePermission>> fileAttributes = PosixFilePermissions.asFileAttribute(perms);
			try {			
				Files.createDirectories(FileSystems.getDefault().getPath(pathToRecreate), fileAttributes);
			} catch (IOException e) {
				LOG.error(e.getMessage(),e);
				return ResponseUtils.sendError("Unable to recreate deleted temp directories. Please check  "+ pathToRecreate);
			}	
			return ResponseUtils.sendError("Recovered from temporary error by recreating temporary directory. Please try to upload logo again.");		
		}
		return ResponseUtils.sendError("Unable to process this request.");
	}

shehzadgee commented Aug 15, 2017

Hi Kappmeier,

I didn't get chance to look into spring boot code to find where this multipart exception is throw. But I handled it my GlobalExceptionHandler in following way. So when I detect that an upload failed because directory is deleted by system cleanup process. I just recreate it and ask user to try upload file again.

So in my humble opinion same check can be placed inside Spring Boot so if directory doesn't exist, it should try to recreate. If it failed due to access rights only than upload should fail.

public  ResponseEntity<?> handle(org.springframework.web.multipart.MultipartException exception) {
		Log.error("handle->MultipartException" + exception.getMessage(),exception);
		// general exception
		if (exception.getCause() instanceof IOException && exception.getCause().getMessage().startsWith("The temporary upload location"))
		{
			String pathToRecreate = exception.getMessage().substring(exception.getMessage().indexOf("[")+1,exception.getMessage().indexOf("]"));
			Set<PosixFilePermission> perms = new HashSet<>();
		    // add permission as rw-r--r-- 644
		    perms.add(PosixFilePermission.OWNER_WRITE);
		    perms.add(PosixFilePermission.OWNER_READ);
		    perms.add(PosixFilePermission.OWNER_EXECUTE);	    
		    perms.add(PosixFilePermission.GROUP_READ);
		    perms.add(PosixFilePermission.GROUP_WRITE);
		    perms.add(PosixFilePermission.GROUP_EXECUTE);
		    FileAttribute<Set<PosixFilePermission>> fileAttributes = PosixFilePermissions.asFileAttribute(perms);
			try {			
				Files.createDirectories(FileSystems.getDefault().getPath(pathToRecreate), fileAttributes);
			} catch (IOException e) {
				LOG.error(e.getMessage(),e);
				return ResponseUtils.sendError("Unable to recreate deleted temp directories. Please check  "+ pathToRecreate);
			}	
			return ResponseUtils.sendError("Recovered from temporary error by recreating temporary directory. Please try to upload logo again.");		
		}
		return ResponseUtils.sendError("Unable to process this request.");
	}

@philwebb philwebb reopened this Aug 15, 2017

@philwebb philwebb added this to the 2.0.0.RC1 milestone Aug 15, 2017

@philwebb

This comment has been minimized.

Show comment
Hide comment
@philwebb

philwebb Aug 15, 2017

Member

I'm still not too sure how we can fix this, but I'm happy to try again in the 2.0.x line. One option might be to periodically update a file in those directories so that they don't get deleted (see this comment). Another idea would be to write a special MultipartResolver that tests if the directly exists before attempting to use it.

Member

philwebb commented Aug 15, 2017

I'm still not too sure how we can fix this, but I'm happy to try again in the 2.0.x line. One option might be to periodically update a file in those directories so that they don't get deleted (see this comment). Another idea would be to write a special MultipartResolver that tests if the directly exists before attempting to use it.

@philwebb philwebb modified the milestones: 2.0.0.RC1, 2.0.0.M6 Sep 20, 2017

@jintaoit

This comment has been minimized.

Show comment
Hide comment
@jintaoit

jintaoit Oct 13, 2017

My program has no Multipart file upload task but it occurs error like:

org.springframework.web.multipart.MultipartException: Could not parse multipart servlet request; nested exception is java.io.IOException:      The temporary upload location [/tmp/tomcat.7104877156386249310.8070/work/Tomcat/localhost/ROOT] is not valid.

why it occurs really make me confused!

jintaoit commented Oct 13, 2017

My program has no Multipart file upload task but it occurs error like:

org.springframework.web.multipart.MultipartException: Could not parse multipart servlet request; nested exception is java.io.IOException:      The temporary upload location [/tmp/tomcat.7104877156386249310.8070/work/Tomcat/localhost/ROOT] is not valid.

why it occurs really make me confused!

@philwebb

This comment has been minimized.

Show comment
Hide comment
@philwebb

philwebb Jan 19, 2018

Member

Another option would be to have a background thread that periodically writes a file to the folder. That would stop it from being deleted and it would fix other errors not related to file upload.

Member

philwebb commented Jan 19, 2018

Another option would be to have a background thread that periodically writes a file to the folder. That would stop it from being deleted and it would fix other errors not related to file upload.

@philwebb philwebb added this to the Icebox milestone Mar 21, 2018

@philwebb philwebb removed the priority: low label Mar 21, 2018

@sooyoung32

This comment has been minimized.

Show comment
Hide comment
@sooyoung32

sooyoung32 May 24, 2018

I encouter the same issue. In my case, i rebooted my application but it did not work and I tried to create the directory (tomcat.xxxx/work/../ROOT) manually it did not work.
Also My app do not have a funtion like uploading file. The exception occurrd when I use restTemplate (post). I do not understand why my spring app use the MultipartServlet..
Please help me out.

sooyoung32 commented May 24, 2018

I encouter the same issue. In my case, i rebooted my application but it did not work and I tried to create the directory (tomcat.xxxx/work/../ROOT) manually it did not work.
Also My app do not have a funtion like uploading file. The exception occurrd when I use restTemplate (post). I do not understand why my spring app use the MultipartServlet..
Please help me out.

@wilkinsona

This comment has been minimized.

Show comment
Hide comment
@wilkinsona

wilkinsona May 24, 2018

Member

@sooyoung32 As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements. If you are looking for some help, please come and chat with the community on Gitter or post a question on Stack Overflow.

Member

wilkinsona commented May 24, 2018

@sooyoung32 As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements. If you are looking for some help, please come and chat with the community on Gitter or post a question on Stack Overflow.

@sooyoung32

This comment has been minimized.

Show comment
Hide comment
@sooyoung32

sooyoung32 May 25, 2018

@wilkinsona Okay. Thank you for the answer. By the way My app work properly now after a day after rebooting my app. I guess the directory was cached in somewhere ...
I posted my question in Stackoverflow. (https://stackoverflow.com/questions/50523407/the-temporary-upload-location-tmp-tomcat-4296537502689403143-5000-work-tomcat)

sooyoung32 commented May 25, 2018

@wilkinsona Okay. Thank you for the answer. By the way My app work properly now after a day after rebooting my app. I guess the directory was cached in somewhere ...
I posted my question in Stackoverflow. (https://stackoverflow.com/questions/50523407/the-temporary-upload-location-tmp-tomcat-4296537502689403143-5000-work-tomcat)

@VitalieS

This comment has been minimized.

Show comment
Hide comment
@VitalieS

VitalieS Jul 26, 2018

I had the exact error on my application today. It said temporary upload location is not valid.

What I did to solve the issue was to relaunch the application adding -java.tmp.dir=/path/to/application/temp/ and creating a /temp/ folder in my application folder.

VitalieS commented Jul 26, 2018

I had the exact error on my application today. It said temporary upload location is not valid.

What I did to solve the issue was to relaunch the application adding -java.tmp.dir=/path/to/application/temp/ and creating a /temp/ folder in my application folder.

@philwebb

This comment has been minimized.

Show comment
Hide comment
@philwebb

philwebb Jul 26, 2018

Member

@VitalieS Yes, the bug is still open. If you look at the very top of the issue there's a little icon that tells you the current status.

Member

philwebb commented Jul 26, 2018

@VitalieS Yes, the bug is still open. If you look at the very top of the issue there's a little icon that tells you the current status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment