using tempdirectory from config/config.php #330
Comments
|
I gues it depends on the preview provider. Are you using a custom third party provider like e.g. Those external binaries might use separate mechanisms for determining a temp directory. Please post a list of your enabled preview providers ( |
Here the enabled preview providers on the machine in question: What makes me ponder though is that the /tmp directory fill up only happens when I start this app and these entries all look the same and are not deleted afterwards. However, when I disable the app and the thumbnails/previews are created "on the fly" when opening a directory, including those for videos etc., /tmp is not used at all. |
|
It is very likely that one of the more complex preview providers is responsible for that. It would be awesome if you could pinpoint the exact file whose previews are causing unwanted files on /tmp. Otherwise, it would be very helpful if you could provide file names of those rogue temp files on /tmp. |
|
Yesterday I have tried it again:
Here two randomly selected ones more thoroughly and in detail: Analizing those files: The strange thing is that for security reasons I disabled preview files to be created for document files. I removed them from the 'enabledPreviewProviders' array. |
|
Now I did this: <?php
$CONFIG = array (
// 'preview_libreoffice_path' => '/usr/bin/libreoffice',
// 'preview_office_cl_parameters' => ' --headless --nologo --nofirststartwizard --invisible --norestore --convert-to png --outdir ',
'enabledPreviewProviders' =>
'enabledPreviewProviders' =>
array (
0 => 'OC\\Preview\\PNG',
1 => 'OC\\Preview\\JPEG',
2 => 'OC\\Preview\\GIF',
3 => 'OC\\Preview\\HEIC',
4 => 'OC\\Preview\\BMP',
5 => 'OC\\Preview\\XBitmap',
6 => 'OC\\Preview\\Krita',
7 => 'OC\\Preview\\WebP',
8 => 'OC\\Preview\\SVG',
9 => 'OC\\Preview\\TIFF',
10 => 'OC\\Preview\\Photoshop',
11=> 'OC\\Preview\\MP3',
12 => 'OC\\Preview\\Movie',
),
'preview_max_x' => 512,
'preview_max_y' => 512,
);Then I removed all previews as described in README.md under "I want to reset/regenerate all previews" and started a fresh per user ./occ preview:generate-all, starting with a user which only have photos and some pdf and office files (from firstrunwizard/skeletondirectory). And again, while scanning those pdf- and office files, the same tmp-directory structures with exactly the same directory name pattern where created:
and again not deleted after processing. Since I have some user accounts with thousends of pdf-files, partially on shared directories, causing the previews to be rescanned for each user with access to those directories, from their account's path selves, this results in a bloated /tmp directory which is (in my case) mounted on tmpfs, ending up in a frozen server. As a workaround I wrote this wrapper script that cleans up everything older than 10 minutes.
#!/bin/bash
# change path and webserver-user accordingly:
nc_dir="/var/www/nextcloud"
log_dir="/var/log/nextcloud"
ht_user="www-data"
occ_call="php -f $nc_dir/occ"
if [ "`id -un`" != "$ht_user" ]; then
echo "This script mus be run as the webserver user \"$ht_user\""
exit 1
fi
cd $nc_dir
logpath="$log_dir/preview-generate-all_"
ask=true
nolog=false
v_flag="-v"
while (( "$#" )); do
case "$1" in
-n|--no-interaction)
ask=false
shift
;;
-vvv|-vv|-v|--verbose)
v_flag="$1"
shift
;;
-q|--quiet|--nolog)
nolog=true
shift
;;
*) echo "unknown argument $1"
exit 1
;;
esac
done
# define trap
kill_background_job() {
echo -e "\nReceived INT signal (^C)"
if [ -n "$BACKGROUND_PID" ]; then
if kill -0 "$BACKGROUND_PID" >/dev/null 2>&1; then
echo "Killing background job with PID $BACKGROUND_PID..."
kill -9 "$BACKGROUND_PID"
fi
fi
wait
exit 1
}
cleanup() {
if find /tmp -maxdepth 1 -user $ht_user -type d -regex ".*/lu[a-z0-9]+\.tmp$" 2>/dev/null | grep -q . ; then
rm -fr /tmp/lu[a-z0-9]*.tmp
echo "temporary files removed by trap"
fi
}
trap "cleanup" EXIT
trap 'kill_background_job' INT
# Define the function to keep the temp directory clean
keep_tmp_clean() {
PROCESS_PID=$1
while ps -p $PROCESS_PID > /dev/null
do
find /tmp -maxdepth 1 -user $ht_user -type d -regex ".*/lu[a-z0-9]+\.tmp$" -mmin +10 -exec rm -rf {} \;
sleep 60
done
# Once the process is done, delete the remaining temp directories
rm -fr /tmp/lu[a-z0-9]*.tmp
}
# We need to take the detour via base64 to allow usernames with spaces
for user in $(echo $($occ_call user:list --output=json | jq -r 'keys_unsorted[] | @base64')); do
user="$(echo "$user" | base64 --decode)"
if $ask; then
echo
read -rp "Press ENTER to generate preview images for user \"$user\", or 's' to skip, or ^C to cancel: " input
if [[ "$input" =~ ^[sS]$ ]]; then
echo "Skipping user \"$user\""
continue
fi
fi
if $nolog; then
echo "Processing user = \"$user\""
$occ_call preview:generate-all --no-interaction "$user" &
BACKGROUND_PID=$!
else
echo -e "User = \"$user\"\nLogfile = \"$logpath$user.log\""
$occ_call preview:generate-all --no-interaction "$v_flag" "$user" >> "$logpath$user.log" &
BACKGROUND_PID=$!
fi
keep_tmp_clean $BACKGROUND_PID
echo "Previews for user \"$user\" generated"
done
# Wait for all background processes to finish
wait
exit 0 |
|
Do you have Nextcloud Office registers its own preview providers. See nextcloud/richdocuments#1335 Related issues in this repo: #410, #419 This will need to get addressed in |
The app does not use the temporary directory defined in the config/config.php nor the one defined in php.ini under 'sys_temp_dir'.
My normal /tmp dir is tmpfs and limited in space. The result after a
occ preview:generate-allwas a server down after the tmp directory was full, until I debugged the problem.Edit: I would actually expect the temp files to be removed by the app as soon as they have done their job.
The text was updated successfully, but these errors were encountered: