Exclude video

[darrepac]

Hi,

It seems that I will have to import again my 40k assets library. It is quite slow and particularly due to video files (transcoding).
Is there a way to:

• exclude video files (in order to speed up first scan) and include it later
• do not transcode video files. I could want to stream directly my video files instead of having a transcoded version of it (and it would speed up the scan by avoiding ffmpeg transcoding)

thanks

[fuji246]

Currently not supported yet, will discuss with team on this.

[fuji246]

We will provide cmd line option to skip video transcoding when import.

[fuji246]

@darrepac , exclude video is added, you can update lomoc to try again.

+				cli.BoolFlag{
+					Name:  "no-video, nv",
+					Usage: "not scan video files. This is to speed up the first import process",
+				},

[darrepac]

Hi, Problem is that even if I stop the container and update it, when I launch it, it starts again the previously stopped process... How can I stop the job and start a new one with the no-video option? thanks Le dim. 28 août 2022 à 10:20, Jiantao Fu ***@***.***> a écrit :

…

@darrepac <https://github.com/darrepac> , exclude video is added, you can update lomoc to try again. + cli.BoolFlag{ + Name: "no-video, nv", + Usage: "not scan video files. This is to speed up the first import process", + }, — Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWMPIVIPIQVNXVLPVOGF5TV3MHNHANCNFSM53J5LOTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dwebfan]

@darrepac sorry to hear that. Can you please describe the problem more? Do you mean restarting container will resume last import automatically?

Importer is a little weak on our side now. It is the top priority now, and will update you shortly.

Regarding this suggestion, just wonder if you use our mobile app to watch video before, and what is your user experience? How is your network speed? and how powerful is your server CPU? Is it enough to download original resolution video? Do you like the idea to do just in time transcoding using ffmpeg while downloading?

• do not transcode video files. I could want to stream directly my video files instead of having a transcoded version of it (and it would speed up the scan by avoiding ffmpeg transcoding)

[darrepac]

@darrepac sorry to hear that. Can you please describe the problem more? Do you mean restarting container will resume last import automatically?

Well, I don't really understand what's going on. I just restarted the container to check the logs.
I see:
time="2022-08-30T18:51:48Z" level=info msg="start generate missed preview"
then a lot of message like this (still counting):
time="2022-08-30T18:52:28Z" level=warning msg="[/media/primary/pascal/Photos/master/2018/07/25/20180725_77850.jpg /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850] - /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp exif analysis got: exiftool analysis /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp got Error: File not found - /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp\n 1 files could not be read\n 0 output files created\n: exit status 1"

[darrepac]

Importer is a little weak on our side now. It is the top priority now, and will update you shortly.

Regarding this suggestion, just wonder if you use our mobile app to watch video before, and what is your user experience? How is your network speed? and how powerful is your server CPU? Is it enough to download original resolution video? Do you like the idea to do just in time transcoding using ffmpeg while downloading?

• do not transcode video files. I could want to stream directly my video files instead of having a transcoded version of it (and it would speed up the scan by avoiding ffmpeg transcoding)

First:
My server is a dell latitude laptop E6410 (intel Core i5 M560 and ssd for database and cache, the original photos being on an hdd).
Obivously it is not the most powerful machine, but it is not a Raspberry either.

Second:
I did not really get the occasion to well test the lomorage app due to 2 main problems: import is running since weeks (so while it is importing I cannot do much things on the server). Could make sense to lower the priority of the import process by the way in order to let the server run is other tasks well. Second problem was my wan network which is quite weak but I can test on my lan network for the moment (I should get a gigabit fiber internet at home the Sept 9).

My suggestion is may-be not well written and should be more written from user point of view instead of technical point of view: lomorage is too too slow importing video. All other solutions I tested were faster (from not transcoding, like plex) to may-be 1 transcoding but lomorage is doing many resolution transcoding... or may-be ffmpeg is less optimized, I don't know.

Hope it helps :)

[darrepac]

@darrepac sorry to hear that. Can you please describe the problem more? Do you mean restarting container will resume last import automatically?

Well, I don't really understand what's going on. I just restarted the container to check the logs. I see: time="2022-08-30T18:51:48Z" level=info msg="start generate missed preview" then a lot of message like this (still counting): time="2022-08-30T18:52:28Z" level=warning msg="[/media/primary/pascal/Photos/master/2018/07/25/20180725_77850.jpg /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850] - /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp exif analysis got: exiftool analysis /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp got Error: File not found - /media/primary/pascal/Photos/preview/2018/07/25/20180725_77850_640_0.webp\n 1 files could not be read\n 0 output files created\n: exit status 1"

I stopped the container waiting for your insights cause I am a bit afraid that it is resetting again all the db…

[dwebfan]

@darrepac Is it possible to tar all log files under /opt/lomorage/var/log, and send to me dwebfan@gmail.com. I guess it is related with the files deleted at original location, but not so sure, so hope to get more logs to analysis. Thanks again.

[darrepac]

There is nothing in /opt/lomorage/var but I sent you by email the log from the app

[fuji246]

It's already added:

1. import command line tool use "--no-video" option to skip video import, https://docs.lomorage.com/docs/Usage/lomorage-client/#import-from-one-existing-folders
2. when running lomod, use --preview-size-video=0 to disable video preview generation , https://github.com/lomorage/lomod#preview-size-video

[darrepac]

👍

…

Le 7 août 2022 à 05:05, Jiantao Fu ***@***.***> a écrit :  We will provide cmd line option to skip video transcoding when import. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[fuji246]

close it now, feel free to reopen if it still happens.