Bug - Video files from mount drives cannot be uploaded

[tissole]

I'm on Windows 10 running uploadgram 0.1.7

I mounted a Google Drive on my PC and tried to upload from there to TG. It works for audios and documents, but sometimes I encounter this error
https://sourceb.in/aw4yDgIKIx

It always fails when I try to upload videos with this error
https://sourceb.in/5rDRPxOc64

I suspected that the cause was those thumbnails generated (maybe no write access for the mount folders?). So I commented all the lines in the script with a reference to thumbnails. Now the script uploads videos with no issues.

Maybe solving the previous issues with thumbnails will fix this problem or a new solution is needed.

[tissole]

Was resolved in #1 (comment)

[xd003]

This is exactly what i wanted . Ability to upload on telegram directly off a mounted drive . Can you please tell what all changes i need to do for this ? ( Also i would like to have thumbnaill enabled for streamable videos )

[tissole]

Sorry for the late response, I didn't see your question earlier. I described the modifications here. You must have FFmpeg installed and in your path, and also you have to install natsort, if you want to have the files/videos sorted in natural order 1,2.3...not 1,10,2. After that you use the script as usual, but instead of the path to the local directory you use the path to folders in the mounted drive.

If you have difficulties, I could upload the modified file. Let me know if you encounter problems. Good luck!

[xd003]

@tissole
Thanks for the reply . Having natsort installed & ffmpeg in path i can do . But the link you sent has references to multiple issues i guess & i am not really familiar with python . I would appreciate if you can just send me the modified files ( or if you have a uploadgram fork ) which supports uploading directly off mount & fixes those other issues that you have mentioned in there as i too face those :)

[tissole]

Don't worry, I'm not familiar at all with python, just hours spent on documenting and trial and error ;) I've changed the time between uploads from 10 to 5 seconds (line 81). You could change it back or try a lower value, but if you go much lower, the flood waits will appear more frequently.

Now the script automatically restarts after the flood waits, you don't have to restart it manually. But even those flood msgs only appear if you go 0s or if you upload many small files in a short period of time. With big files, they don't appear.

Here is the file, just replace the old file in uploadgram folder. Let me know if it is working!
upload.zip

[xd003]

Great , i am gonna try this now . Just to be sure , while uploading to telegram off mount the script doesn't downloads the files locally in any way right ? Like say with system having just 10 gb of storage , i can upload files of any size whatsoever right ? ( Eg say i have 50 files of 2 gb each total being 100 gb )

[tissole]

No, I haven't seen any files downloaded locally. It happens in memory, I think.

[xd003]

@tissole Hello , i couldn't test uploadgram with your modified file as i wasn't home this week . Finally got around testing it but unfortunately it doesn't seem to work . After i run the uploadgram command , it starts the process and ends it automatically in 1-2 seconds . i am using rclone mount to mount my google drive remote

Also seeing the changes in your file , only difference i can notice is the floodwait & natsort thing . Not really getting how could that make mount upload to start working

[tissole]

There are more changes there, some critical to uploading from mounted drives. You replaced the original file with the modified one, or you modified the original file? What is the message error and what command did you run?

[xd003]

@tissole
I replaced yours upload.py in my forked repo - commit here & then installled this uploadgram fork with pip ( you may check to confirm nothing is missing from this file )

Rclone command used to mount - rclone mount remote: /GD --log-file $HOME/rclone/mount.log --log-level DEBUG --umask 002 --poll-interval 15s --vfs-cache-mode writes --allow-other

Uploadgram command used - uploadgram -100XXXXXXXXXX $FILE

There's not really any error , after running the uploadgram command it starts normally & ends on its own normally like it does ( but all this completes within 2-3 seconds so there isn't anything uploaded )

[tissole]

Try uploadgram -100XXXXXXXXXX path-to-mounted-drive

[xd003]

Try uploadgram -100XXXXXXXXXX path-to-mounted-drive

Ah yes , finally it worked . Not sure if the file itself was corrupted or something else . Its working for all files on drive mount now

Also just to confirm , is it just me or the upload through mount is 2,3 times slower ?
I have a 2 gigabit server , generally i get around 30-40 MBPS speed if uploading files from local storage ( still not really that fast sadly :/) , with mount its hovering around 10-12

[tissole]

Yes, it is a little bit slower when uploading from a mounted drive. You can also drag and drop from a mounted drive directly in to a chat window open in a telegram client. You can use Kotatogram or 64Gram , these clients have a setting for boosting upload speed.

[xd003]

yeah i have heard about these clients but if each of our files to be uploaded is inside a folder than we can't multi select them so its not feasible to do it from client GUI , with cli i can just use uploadgram to loop over each folder . i wish we could have these speed enhancements from 64gram with cli on uploadgram itself , do you know the relevant part of 64gram code which achieves this ? i can probably get it done from some people who know pyrogram well

[tissole]

Uploadgram is much faster than telegram-upload, because it is based on Pyrogram. The original project uses Telethon which uploads files sequentially. I think Pyrogram uses parallel uploads.
The code is probably 1 and 2, 3, 4
There is also FastTelethon
If you create something new, make a pull request for Uploadgram.

[xd003]

That's great , i will check the links you sent to check the relevent modifications in 64gram to see if it can be done with pyrogram to implement in uploadgram.
Yes you are correct , telegram-upload is so horribly slow , for me its almost unusable ( almost 10-20x slower than uploadgram )

On another note , have you been able to test speed of FastTelethon ? ( Is it faster than uploadgram ) I am really eager to test it out but just can't figure out how to use it

[tissole]

I have tested FastTelethon, Pyrogram is still fastest.
If you want to test Telethon vs. Pyrogram use Telebench (req. are telethon, pyrogram, cryptg and tgcrypto for fast encryption/decryption). Put a file to upload in the same folder with the script, modify line 9 accordingly, and in line 10 put your username. Ignore the stats from console, I don't find it accurate, check actual speed with task manager. The file is uploaded in the bot chat.

For FastTelethon use url-uploader. Install req., export envs and run python main.py. Then go to the bot and run/upload https://dl-cdn.alpinelinux.org/alpine/v3.16/releases/x86_64/alpine-extended-3.16.2-x86_64.iso This will download a file and then upload it to bot chat. Watch the speed in task manager.