Disk space leak

[Dannecron]

Hello. I ran into another disk space leak issue. Unlike this issue #87, directories stay in /tmp inside container after every convertation request (office endpoint, docx to pdf).

Expected Behavior

There is no disc space leak, after starting containter.

Current Behavior

After creation directories inside /tmp stays forever.

Steps to Reproduce (for bugs)

1. Run gotenberg container
2. Send request to office endpoint with single docx doc inside

Your Environment

• Version used: 6.1.1, 6.2.0
• Envs:
  • DISABLE_GOOGLE_CHROME: 1
  • DEFAULT_WAIT_TIMEOUT: 20
• Docker version: 19.03.8, 19.03.5
• also tried with docker-compose 1.22.0

Logs

gotenberg_1   | {"level":"info","msg":"Gotenberg 6.2.0","op":"main","time":"2020-04-24T03:35:52Z","trace":"system"}
gotenberg_1   | {"level":"debug","msg":"configuration: {maximumWaitTimeout:30 maximumWaitDelay:10 maximumWebhookURLTimeout:30 defaultWaitTimeout:20 defaultWebhookURLTimeout:10 defaultListenPort:3000 disableGoogleChrome:true disableUnoconv:false logLevel:DEBUG rootPath:/ maximumGoogleChromeRpccBufferSize:104857600 defaultGoogleChromeRpccBufferSize:1048576}","op":"main","time":"2020-04-24T03:35:52Z","trace":"system"}
gotenberg_1   | {"level":"info","msg":"http server started on port '3000'","op":"main","time":"2020-04-24T03:35:52Z","trace":"system"}
gotenberg_1   | {"level":"debug","msg":"resource directory 'twl82HeGJF1nGEWCiozhTqYOd1s6Y55G' created","op":"resource.New","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"skipping 'Content-Length' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"skipping 'User-Agent' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"skipping 'Content-Type' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'resultFilename' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'waitTimeout' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'webhookURL' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'webhookURLTimeout' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'remoteURL' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'waitDelay' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'paperWidth' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'paperHeight' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'marginTop' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'marginBottom' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'marginLeft' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'marginRight' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'landscape' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'pageRanges' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'googleChromeRpccBufferSize' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"added 'scale' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"resource file 'doc5ea25f3c21174.docx' created","op":"resource.Resource.WithFile","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"handling Office request...","op":"xhttp.officeHandler","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"no 'webhookURL' found, converting synchronously","op":"xhttp.convert","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"options: {WaitTimeout:20 Landscape:false PageRanges:}","op":"printer.logOptions","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"creating context with '20.00s' of timeout...","op":"xcontext.WithTimeout","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"converting '/gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G/doc5ea25f3c21174.docx' to PDF...","op":"printer.officePrinter.Print","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"executing command: unoconv --user-profile ///tmp/39929 --port 39929 --format pdf --output /gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G/0kwKgDq2w4fNxS4LWaCiRfR3adB2qZOdm.pdf /gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G/doc5ea25f3c21174.docx","op":"xexec.LogBeforeExecute","time":"2020-04-24T03:38:36Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"command 'unoconv --user-profile ///tmp/39929 --port 39929 --format pdf --output /gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G/0kwKgDq2w4fNxS4LWaCiRfR3adB2qZOdm.pdf /gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G/doc5ea25f3c21174.docx' finished","op":"xexec.Run","time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"'kwKgDq2w4fNxS4LWaCiRfR3adB2qZOdm.pdf' created","op":"printer.officePrinter.Print","time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"only one PDF created, nothing to merge","op":"printer.officePrinter.Print","time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"no 'remoteURL' found, using generated filename 'drth2p3Gfc6wzNv4kknpDXucyW8JaFiC.pdf'","op":"xhttp.convertSync","time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"level":"debug","msg":"resource directory '/gotenberg/tmp/twl82HeGJF1nGEWCiozhTqYOd1s6Y55G' removed","op":"resource.Resource.Close","time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G"}
gotenberg_1   | {"bytes_in":"72559","bytes_out":"167689","host":"gotenberg:3000","latency":"3444551068","latency_human":"3.444551068s","level":"info","method":"POST","msg":"request handled","op":"context.Context.LogRequestResult","path":"/convert/office","referer":"","remote_ip":"172.22.0.10","status":200,"time":"2020-04-24T03:38:39Z","trace":"twl82HeGJF1nGEWCiozhTqYOd1s6Y55G","uri":"/convert/office","user_agent":"GuzzleHttp/6.3.3 curl/7.64.0 PHP/7.3.13"}
gotenberg_1   | {"level":"debug","msg":"resource directory 'bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9' created","op":"resource.New","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"skipping 'Content-Type' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"skipping 'Content-Length' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"skipping 'User-Agent' as it is not a custom HTTP header...","op":"resource.Resource.WithCustomHTTPHeader","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'resultFilename' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'waitTimeout' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'webhookURL' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'webhookURLTimeout' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'remoteURL' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'waitDelay' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'paperWidth' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'paperHeight' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'marginTop' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'marginBottom' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'marginLeft' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'marginRight' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'landscape' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'pageRanges' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'googleChromeRpccBufferSize' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"added 'scale' with value '' to resource args","op":"resource.Resource.WithArg","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"resource file 'doc5ea25fbc317dd.docx' created","op":"resource.Resource.WithFile","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"handling Office request...","op":"xhttp.officeHandler","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"no 'webhookURL' found, converting synchronously","op":"xhttp.convert","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"options: {WaitTimeout:20 Landscape:false PageRanges:}","op":"printer.logOptions","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"creating context with '20.00s' of timeout...","op":"xcontext.WithTimeout","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"converting '/gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9/doc5ea25fbc317dd.docx' to PDF...","op":"printer.officePrinter.Print","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"executing command: unoconv --user-profile ///tmp/33395 --port 33395 --format pdf --output /gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9/0SNxBrwMWDUjXU70XRam2bhpzcXsd1MKj.pdf /gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9/doc5ea25fbc317dd.docx","op":"xexec.LogBeforeExecute","time":"2020-04-24T03:40:44Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"command 'unoconv --user-profile ///tmp/33395 --port 33395 --format pdf --output /gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9/0SNxBrwMWDUjXU70XRam2bhpzcXsd1MKj.pdf /gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9/doc5ea25fbc317dd.docx' finished","op":"xexec.Run","time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"'SNxBrwMWDUjXU70XRam2bhpzcXsd1MKj.pdf' created","op":"printer.officePrinter.Print","time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"only one PDF created, nothing to merge","op":"printer.officePrinter.Print","time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"no 'remoteURL' found, using generated filename 'hSjRZ1c0dp9PHU0x9ErysgZ0DJWMhEPR.pdf'","op":"xhttp.convertSync","time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"level":"debug","msg":"resource directory '/gotenberg/tmp/bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9' removed","op":"resource.Resource.Close","time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9"}
gotenberg_1   | {"bytes_in":"23509","bytes_out":"80842","host":"gotenberg:3000","latency":"2403301881","latency_human":"2.403301881s","level":"info","method":"POST","msg":"request handled","op":"context.Context.LogRequestResult","path":"/convert/office","referer":"","remote_ip":"172.22.0.10","status":200,"time":"2020-04-24T03:40:46Z","trace":"bPgEAAkJRvdnTZxMux6u1JtQH1L10jo9","uri":"/convert/office","user_agent":"GuzzleHttp/6.3.3 curl/7.64.0 PHP/7.3.13"}

gotenberg@3d51317c20b1:~$ ls -la /tmp
total 20
drwxrwxrwt 1 root      root      4096 Apr 24 03:40 .
drwxr-xr-x 1 root      root      4096 Apr 24 03:35 ..
drwx------ 3 gotenberg gotenberg 4096 Apr 24 03:40 33395
drwx------ 3 gotenberg gotenberg 4096 Apr 24 03:38 39929
drwxr-xr-x 2 root      root      4096 Feb 10 09:25 hsperfdata_root

[gulien]

Hello @Dannecron,

Thanks for the well written issue.

What's happening?

For allowing concurrent conversions with LibreOffice, you have to specify a user profile directory and a non-used port. For instance:

unoconv --user-profile ///tmp/33395 --port 33395 --format pdf [...]

As shown, that's exactly how Gotenberg does it by creating a user profile directory based on the port used by the conversion.

But what it should also do is removing this user profile directory once the conversion is done (https://github.com/thecodingmachine/gotenberg/blob/master/internal/pkg/printer/office.go#L116).

I'll try to find some time to fix this issue, but if you could open a PR that would be great 😄

[gulien]

Fixed in version 6.2.1. It should be available on Docker Hub soon.

[gulien]

When an Office conversion is terminated because of a context timeout, there are some artifacts remaining in the /tmp folder.

Under heavy load, the application seems to struggle to remove user profile directories.