Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Disk Usage ; #12

Closed
akhilleusuggo opened this issue Jun 30, 2020 · 17 comments
Closed

Update Disk Usage ; #12

akhilleusuggo opened this issue Jun 30, 2020 · 17 comments

Comments

@akhilleusuggo
Copy link
Contributor

When I update disk usage , does not report the right usage . For some reason does it multiple times . Those are the logs of after 1 click on disk update usage .

[30/Jun/2020:08:52:54 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"
[30/Jun/2020:08:52:59 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_Low.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_SD.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_Low.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_SD.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_Low.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_SD.mp4 HTTP/1.1" 200 3645 "-" "-"
 [30/Jun/2020:08:53:01 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"

File size shown on MyVideos section :
image

Real file size :
image
@DanielnetoDotCom
Copy link
Member

this is really strange.
does it happen every time you click on it? or just some times.
I was not able to reproduce it here.

@akhilleusuggo
Copy link
Contributor Author

everytime .
I mean this gonna be an issue for many people who's planning to get the DiskQuota plugin .

@DanielnetoDotCom
Copy link
Member

Agree this is wrong, I just cannot see how is it happening.

is it a video hosted in the storage?

the same happens with local videos?

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 3, 2020 •
edited
Loading

@DanielnetoDotCom happens much more on storage site then the local site .
I mean right now , all my storage site is wrong . Everything indicates x2 x3 more size then it is .

I'm not sure if this is caused by the dump files on the streamer site .

[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179/index.m3u8 in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179.mp4 in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179.webm in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179_Low.webm in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179_SD.webm in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179_HD.webm in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179.pdf in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179.zip in /var/www/html/YouPHPTube/objects/video.php on line 2575
[03-Jul-2020 15:12:54 Europe/Helsinki] PHP Warning:  filesize(): stat failed for /var/www/html/YouPHPTube/videos/_ID.81107179.mp3 in /var/www/html/YouPHPTube/objects/video.php on line 2575

Then it's start some curl for header .
But again , does that x2 x3 times . Witch explains why does report wrong filesize .
Because the dump files does redirect to the storage site . So it's counting the dump files as files .

@DanielnetoDotCom
Copy link
Member

It will count the dummy files, but it supposes to be only 10 bytes each.
also count the images, and zip files if they were not deleted.
can you please check if it deletes the zip files?

I mean it will count anything with the video "filename" form database

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 3, 2020 •
edited
Loading

image

No zip files , nothing except dump files.

@DanielnetoDotCom
Copy link
Member

can you please make something like this in your streamer and storage

ls -lah /var/www/html/YouPHPTube/videos/_ID.81107179*

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 3, 2020 •
edited
Loading

STORAGE SITE :
image

STREAMER SITE:
image

@DanielnetoDotCom
Copy link
Member

Ok, all looks perfect, so if you click on the "update disk storage" it will give you the wrong size?

image

@akhilleusuggo
Copy link
Contributor Author

Yes Sir

@DanielnetoDotCom
Copy link
Member

ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_Low.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_SD.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_Low.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:00 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_SD.mp4 HTTP/1.1" 200 3645 "-" "-"
[30/Jun/2020:08:53:01 +0000] "HEAD /videos/_YPTuniqid_5ef852e370b841.68237251_HD.mp4 HTTP/1.1" 200 3646 "

Is this log from storage or streamer?

Could you detect if is the streamer that is requesting multiple times or is the storage that is processing multiple times?

@DanielnetoDotCom
Copy link
Member

I will fix, I could reproduce it here

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 3, 2020 •
edited
Loading

Hello Daniel , on the streamer site , too many requests are made that looks like inspecting the header

[03-Jul-2020 15:38:29 Europe/Helsinki] AVideoLog::DEBUG: getUsageFromURL: start (https://storage.example.com/videos/_ID.68237251_HD.mp4)
[03-Jul-2020 15:38:29 Europe/Helsinki] AVideoLog::DEBUG: getUsageFromURL: curl_init 
[03-Jul-2020 15:38:29 Europe/Helsinki] AVideoLog::DEBUG: getUsageFromURL: response header HTTP/1.1 200 OK
Date: Fri, 03 Jul 2020 12:38:29 GMT
Server: Apache
Set-Cookie: 46545649a030c1c8227=srpekdqtov1654654654jt0vj6; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: max-age=1, private, must-revalidate
Pragma: no-cache
Set-Cookie: 79f9a78441fe287987930c1c8227=srpekdqtov16543219b7jt0vj6; path=/
Access-Control-Allow-Origin: *
Last-Modified: Mon, 29 Jun 2020 13:49:14 GMT
ETag: "11b31ec4e-fb8a80"
Content-Length: 4751223886
Content-Type: video/mp4

I don't wanna past here all the log , but you got the idea , just too many of this requests of header .

DanielnetoDotCom added a commit to WWBN/AVideo that referenced this issue Jul 3, 2020
@DanielnetoDotCom
https://github.com/WWBN/AVideo-Storage/issues/12
945890e
@DanielnetoDotCom
Copy link
Member

Please try this update on the Streamer site

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 3, 2020 •
edited
Loading

Works for me , but php updateDiskUsage not working .
Nor
image

@akhilleusuggo
Copy link
Contributor Author

When running php updateDiskUsage
on /install directory

image

@akhilleusuggo
Copy link
Contributor Author

akhilleusuggo commented Jul 4, 2020 •
edited
Loading

@DanielnetoDotCom Daniel , I have to delete everyday handreds of this files . Can you send a fix ? Maybe they should be on /tmp directory and not on the /object directory or /install

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@DanielnetoDotCom @akhilleusuggo