Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get filesize #190

Closed
samrat opened this issue Oct 9, 2011 · 7 comments
Closed

get filesize #190

samrat opened this issue Oct 9, 2011 · 7 comments
Labels

Comments

@samrat
Copy link

samrat commented Oct 9, 2011

Since the file size is known before the download completes, an option could be added to find the filesize similar to --get-filename.

@phihag
Copy link
Contributor

phihag commented Oct 10, 2011

While the filesize is known while downloading, it is generally not known before downloading. We'll need to send a HEAD request in these cases then.

@samrat
Copy link
Author

samrat commented Oct 11, 2011

@phihag - Can I find the filesize without downloading then? Could you please show me how to do it? I really need this.

@sivaramn
Copy link

Second the feature request for --get-filesize option. In lots of places in the developing world, bandwidth is throttled after the first 5Gb/10Gb etc, it would make sense to check the file one's downloading before fetching.

Right, I understand that there's a technical issue that you'd have to initiate the download before getting to know the file size but still, it would be an option that the user can try at his own risk.

and is there a possibility of --random-wait like wget? That would help in not getting banned accidently. :-)

@yasoob
Copy link
Contributor

yasoob commented Jul 8, 2013

@samrat check out this gist https://gist.github.com/yasoob/9f5528b97dd4522b48e4 I have outlined a method. There is another method as well if you need it then i will upload it as well. But keep in mind that currently it is possible only for http or https videos not for rtmp. :)

@shirishag75
Copy link

interested in it as well hence commenting.

@xanadu
Copy link
Contributor

xanadu commented Aug 30, 2013

You guys will find out a lot of sites are not friendly with HEAD request.

@ghost
Copy link

ghost commented Jun 18, 2014

Same as #6.

@ghost ghost closed this as completed Jun 18, 2014
blackjack4494 referenced this issue in blackjack4494/youtube-dlc Sep 12, 2020
* [scrippsnetworks] Add new extractor(closes ytdl-org#19857)(closes ytdl-org#22981)

* [teachable] Improve locked lessons detection (ytdl-org#23528)

* [teachable] Fail with error message if no video URL found

* [extractors] add missing import for ScrippsNetworksIE

* [brightcove] cache brightcove player policy keys

* [prosiebensat1] improve geo restriction handling(closes ytdl-org#23571)

* [soundcloud] automatically update client id on failing requests

* [spankbang] Fix extraction (closes ytdl-org#23307, closes ytdl-org#23423, closes ytdl-org#23444)

* [spankbang] Improve removed video detection (ytdl-org#23423)

* [brightcove] update policy key on failing requests

* [pornhub] Fix extraction and add support for m3u8 formats (closes ytdl-org#22749, closes ytdl-org#23082)

* [pornhub] Improve locked videos detection (closes ytdl-org#22449, closes ytdl-org#22780)

* [brightcove] invalidate policy key cache on failing requests

* [soundcloud] fix client id extraction for non fatal requests

* [ChangeLog] Actualize
[ci skip]

* [devscripts/create-github-release] Switch to using PAT for authentication

Basic authentication will be deprecated soon

* release 2020.01.01

* [redtube] Detect private videos (ytdl-org#23518)

* [vice] improve extraction(closes ytdl-org#23631)

* [devscripts/create-github-release] Remove unused import

* [wistia] improve format extraction and extract subtitles(closes ytdl-org#22590)

* [nrktv:seriebase] Fix extraction (closes ytdl-org#23625) (ytdl-org#23537)

* [discovery] fix anonymous token extraction(closes ytdl-org#23650)

* [scrippsnetworks] add support for www.discovery.com videos

* [scrippsnetworks] correct test case URL

* [dctp] fix format extraction(closes ytdl-org#23656)

* [pandatv] Remove extractor (ytdl-org#23630)

* [naver] improve extraction

- improve geo-restriction handling
- extract automatic captions
- extract uploader metadata
- extract VLive HLS formats

* [naver] improve metadata extraction

* [cloudflarestream] improve extraction

- add support for bytehighway.net domain
- add support for signed URLs
- extract thumbnail

* [cloudflarestream] import embed URL extraction

* [lego] fix extraction and extract subtitle(closes ytdl-org#23687)

* [safari] Fix kaltura session extraction (closes ytdl-org#23679) (ytdl-org#23670)

* [orf:fm4] Fix extraction (ytdl-org#23599)

* [orf:radio] Clean description and improve extraction

* [twitter] add support for promo_video_website cards(closes ytdl-org#23711)

* [vodplatform] add support for embed.kwikmotion.com domain

* [ndr:base:embed] Improve thumbnails extraction (closes ytdl-org#23731)

* [canvas] Add support for new API endpoint and update tests (closes ytdl-org#17680, closes ytdl-org#18629)

* [travis] Add flake8 job (ytdl-org#23720)

* [yourporn] Fix extraction (closes ytdl-org#21645, closes ytdl-org#22255, closes ytdl-org#23459)

* [ChangeLog] Actualize
[ci skip]

* release 2020.01.15

* [soundcloud] Restore previews extraction (closes ytdl-org#23739)

* [orf:tvthek] Improve geo restricted videos detection (closes ytdl-org#23741)

* [zype] improve extraction

- extract subtitles(closes ytdl-org#21258)
- support URLs with alternative keys/tokens(ytdl-org#21258)
- extract more metadata

* [americastestkitchen] fix extraction

* [nbc] add support for nbc multi network URLs(closes ytdl-org#23049)

* [ard] improve extraction(closes ytdl-org#23761)

- simplify extraction
- extract age limit and series
- bypass geo-restriction

* [ivi:compilation] Fix entries extraction (closes ytdl-org#23770)

* [24video] Add support for 24video.vip (closes ytdl-org#23753)

* [businessinsider] Fix jwplatform id extraction (closes ytdl-org#22929) (ytdl-org#22954)

* [ard] add a missing condition

* [azmedien] fix extraction(closes ytdl-org#23783)

* [voicerepublic] fix extraction

* [stretchinternet] fix extraction(closes ytdl-org#4319)

* [youtube] Fix sigfunc name extraction (closes ytdl-org#23819)

* [ChangeLog] Actualize
[ci skip]

* release 2020.01.24

* [soundcloud] imporve private playlist/set tracks extraction

ytdl-org#3707 (comment)

* [svt] fix article extraction(closes ytdl-org#22897)(closes ytdl-org#22919)

* [svt] fix series extraction(closes ytdl-org#22297)

* [viewlift] improve extraction

- fix extraction(closes ytdl-org#23851)
- add add support for authentication
- add support for more domains

* [vimeo] fix album extraction(closes ytdl-org#23864)

* [tva] Relax _VALID_URL (closes ytdl-org#23903)

* [tv5mondeplus] Fix extraction (closes ytdl-org#23907, closes ytdl-org#23911)

* [twitch:stream] Lowercase channel id for stream request (closes ytdl-org#23917)

* [sportdeutschland] Update to new sportdeutschland API

They switched to SSL, but under a different host AND path...
Remove the old test cases because these videos have become unavailable.

* [popcorntimes] Add extractor (closes ytdl-org#23949)

* [thisoldhouse] fix extraction(closes ytdl-org#23951)

* [toggle] Add support for mewatch.sg (closes ytdl-org#23895) (ytdl-org#23930)

* [compat] Introduce compat_realpath (refs ytdl-org#23991)

* [update] Fix updating via symlinks (closes ytdl-org#23991)

* [nytimes] improve format sorting(closes ytdl-org#24010)

* [abc:iview] Support 720p (ytdl-org#22907) (ytdl-org#22921)

* [nova:embed] Fix extraction (closes ytdl-org#23672)

* [nova:embed] Improve (closes ytdl-org#23690)

* [nova] Improve extraction (refs ytdl-org#23690)

* [jpopsuki] Remove extractor (closes ytdl-org#23858)

* [YoutubeDL] Fix playlist entry indexing with --playlist-items (closes ytdl-org#10591, closes ytdl-org#10622)

* [test_YoutubeDL] Fix get_ids

* [test_YoutubeDL] Add tests for ytdl-org#10591 (closes ytdl-org#23873)

* [24video] Add support for porn.24video.net (closes ytdl-org#23779, closes ytdl-org#23784)

* [npr] Add support for streams (closes ytdl-org#24042)

* [ChangeLog] Actualize
[ci skip]

* release 2020.02.16

* [tv2dk:bornholm:play] Fix extraction (ytdl-org#24076)

* [imdb] Fix extraction (closes ytdl-org#23443)

* [wistia] Add support for multiple generic embeds (closes ytdl-org#8347, closes ytdl-org#11385)

* [teachable] Add support for multiple videos per lecture (closes ytdl-org#24101)

* [pornhd] Fix extraction (closes ytdl-org#24128)

* [options] Remove duplicate short option -v for --version (ytdl-org#24162)

* [extractor/common] Convert ISM manifest to unicode before processing on python 2 (ytdl-org#24152)

* [YoutubeDL] Force redirect URL to unicode on python 2

* Remove no longer needed compat_str around geturl

* [youjizz] Fix extraction (closes ytdl-org#24181)

* [test_subtitles] Remove obsolete test

* [zdf:channel] Fix tests

* [zapiks] Fix test

* [xtube] Fix metadata extraction (closes ytdl-org#21073, closes ytdl-org#22455)

* [xtube:user] Fix test

* [telecinco] Fix extraction (refs ytdl-org#24195)

* [telecinco] Add support for article opening videos

* [franceculture] Fix extraction (closes ytdl-org#24204)

* [xhamster] Fix extraction (closes ytdl-org#24205)

* [ChangeLog] Actualize
[ci skip]

* release 2020.03.01

* [vimeo] Fix subtitles URLs (ytdl-org#24209)

* [servus] Add support for new URL schema (closes ytdl-org#23475, closes ytdl-org#23583, closes ytdl-org#24142)

* [youtube:playlist] Fix tests (closes ytdl-org#23872) (ytdl-org#23885)

* [peertube] Improve extraction

* [peertube] Fix issues and improve extraction (closes ytdl-org#23657)

* [pornhub] Improve title extraction (closes ytdl-org#24184)

* [vimeo] fix showcase password protected video extraction(closes ytdl-org#24224)

* [youtube] Fix age-gated videos support without login (closes ytdl-org#24248)

* [youtube] Fix tests

* [ChangeLog] Actualize
[ci skip]

* release 2020.03.06

* [nhk] update API version(closes ytdl-org#24270)

* [youtube] Improve extraction in 429 error conditions (closes ytdl-org#24283)

* [youtube] Improve age-gated videos extraction in 429 error conditions (refs ytdl-org#24283)

* [youtube] Remove outdated code

Additional get_video_info requests don't seem to provide any extra itags any longer

* [README.md] Clarify 429 error

* [pornhub] Add support for pornhubpremium.com (ytdl-org#24288)

* [utils] Add support for cookies with spaces used instead of tabs

* [ChangeLog] Actualize
[ci skip]

* release 2020.03.08

* Revert "[utils] Add support for cookies with spaces used instead of tabs"

According to [1] TABs must be used as separators between fields.
Files produces by some tools with spaces as separators are considered
malformed.

1. https://curl.haxx.se/docs/http-cookies.html

This reverts commit cff99c9.

* [utils] Add reference to cookie file format

* Revert "[vimeo] fix showcase password protected video extraction(closes ytdl-org#24224)"

This reverts commit 12ee431.

* [nhk] Relax _VALID_URL (ytdl-org#24329)

* [nhk] Remove obsolete rtmp formats (closes ytdl-org#24329)

* [nhk] Update m3u8 URL and use native hls (ytdl-org#24329)

* [ndr] Fix extraction (closes ytdl-org#24326)

* [xtube] Fix formats extraction (closes ytdl-org#24348)

* [xtube] Fix typo

* [hellporno] Fix extraction (closes ytdl-org#24399)

* [cbc:watch] Add support for authentication

* [cbc:watch] Fix authenticated device token caching (closes ytdl-org#19160)

* [soundcloud] fix download url extraction(closes ytdl-org#24394)

* [limelight] remove disabled API requests(closes ytdl-org#24255)

* [bilibili] Add support for new URL schema with BV ids (closes ytdl-org#24439, closes ytdl-org#24442)

* [bilibili] Add support for player.bilibili.com (closes ytdl-org#24402)

* [teachable] Extract chapter metadata (closes ytdl-org#24421)

* [generic] Look for teachable embeds before wistia

* [teachable] Update upskillcourses domain

New version does not use teachable platform any longer

* [teachable] Update gns3 domain

* [teachable] Update test

* [ChangeLog] Actualize
[ci skip]

* [ChangeLog] Actualize
[ci skip]

* release 2020.03.24

* [spankwire] Fix extraction (closes ytdl-org#18924, closes ytdl-org#20648)

* [spankwire] Add support for generic embeds (refs ytdl-org#24633)

* [youporn] Add support form generic embeds

* [mofosex] Add support for generic embeds (closes ytdl-org#24633)

* [tele5] Fix extraction (closes ytdl-org#24553)

* [extractor/common] Skip malformed ISM manifest XMLs while extracting ISM formats (ytdl-org#24667)

* [tv4] Fix ISM formats extraction (closes ytdl-org#24667)

* [twitch:clips] Extend _VALID_URL (closes ytdl-org#24290) (ytdl-org#24642)

* [motherless] Fix extraction (closes ytdl-org#24699)

* [nova:embed] Fix extraction (closes ytdl-org#24700)

* [youtube] Skip broken multifeed videos (closes ytdl-org#24711)

* [soundcloud] Extract AAC format

* [soundcloud] Improve AAC format extraction (closes ytdl-org#19173, closes ytdl-org#24708)

* [thisoldhouse] Fix video id extraction (closes ytdl-org#24548)

Added support for:
with of without "www."
and either  ".chorus.build" or ".com"

It now validated correctly on older URL's
```
<iframe src="https://thisoldhouse.chorus.build/videos/zype/5e33baec27d2e50001d5f52f
```
and newer ones
```
<iframe src="https://www.thisoldhouse.com/videos/zype/5e2b70e95216cc0001615120
```

* [thisoldhouse] Improve video id extraction (closes ytdl-org#24549)

* [youtube] Fix DRM videos detection (refs ytdl-org#24736)

* [options] Clarify doc on --exec command (closes ytdl-org#19087) (ytdl-org#24883)

* [prosiebensat1] Improve extraction and remove 7tv.de support (ytdl-org#24948)

* [prosiebensat1] Extract series metadata

* [tenplay] Relax _VALID_URL (closes ytdl-org#25001)

* [tvplay] fix Viafree extraction(closes ytdl-org#15189)(closes ytdl-org#24473)(closes ytdl-org#24789)

* [yahoo] fix GYAO Player extraction and relax title URL regex(closes ytdl-org#24178)(closes ytdl-org#24778)

* [youtube] Use redirected video id if any (closes ytdl-org#25063)

* [youtube] Improve player id extraction and add tests

* [extractor/common] Extract multiple JSON-LD entries

* [crunchyroll] Fix and improve extraction (closes ytdl-org#25096, closes ytdl-org#25060)

* [ChangeLog] Actualize
[ci skip]

* release 2020.05.03

* [puhutv] Remove no longer available HTTP formats (closes ytdl-org#25124)

* [utils] Improve cookie files support

+ Add support for UTF-8 in cookie files
* Skip malformed cookie file entries instead of crashing (invalid entry len, invalid expires at)

* [dailymotion] Fix typo

* [compat] Introduce compat_cookiejar_Cookie

* [extractor/common] Use compat_cookiejar_Cookie for _set_cookie (closes ytdl-org#23256, closes ytdl-org#24776)

To always ensure cookie name and value are bytestrings on python 2.

* [orf] Add support for more radio stations (closes ytdl-org#24938) (ytdl-org#24968)

* [uol] fix extraction(closes ytdl-org#22007)

* [downloader/http] Finish downloading once received data length matches expected

Always do this if possible, i.e. if Content-Length or expected length is known, not only in test.
This will save unnecessary last extra loop trying to read 0 bytes.

* [downloader/http] Request last data block of exact remaining size

Always request last data block of exact size remaining to download if possible not the current block size.

* [iprima] Improve extraction (closes ytdl-org#25138)

* [youtube] Improve signature cipher extraction (closes ytdl-org#25188)

* [ChangeLog] Actualize
[ci skip]

* release 2020.05.08

* [spike] fix Bellator mgid extraction(closes ytdl-org#25195)

* [bbccouk] PEP8

* [mailru] Fix extraction (closes ytdl-org#24530) (ytdl-org#25239)

* [README.md] flake8 HTTPS URL (ytdl-org#25230)

* [youtube] Add support for yewtu.be (ytdl-org#25226)

* [soundcloud] reduce API playlist page limit(closes ytdl-org#25274)

* [vimeo] improve format extraction and sorting(closes ytdl-org#25285)

* [redtube] Improve title extraction (ytdl-org#25208)

* [indavideo] Switch to HTTPS for API request (ytdl-org#25191)

* [utils] Fix file permissions in write_json_file (closes ytdl-org#12471) (ytdl-org#25122)

* [redtube] Improve formats extraction and extract m3u8 formats (closes ytdl-org#25311, closes ytdl-org#25321)

* [ard] Improve _VALID_URL (closes ytdl-org#25134) (ytdl-org#25198)

* [giantbomb] Extend _VALID_URL (ytdl-org#25222)

* [postprocessor/ffmpeg] Embed series metadata with --add-metadata

* [youtube] Add support for more invidious instances (ytdl-org#25417)

* [ard:beta] Extend _VALID_URL (closes ytdl-org#25405)

* [ChangeLog] Actualize
[ci skip]

* release 2020.05.29

* [jwplatform] Improve embeds extraction (closes ytdl-org#25467)

* [periscope] Fix untitled broadcasts (ytdl-org#25482)

* [twitter:broadcast] Add untitled periscope broadcast test

* [malltv] Add support for sk.mall.tv (ytdl-org#25445)

* [brightcove] Fix subtitles extraction (closes ytdl-org#25540)

* [brightcove] Sort imports

* [twitch] Pass v5 accept header and fix thumbnails extraction (closes ytdl-org#25531)

* [twitch:stream] Fix extraction (closes ytdl-org#25528)

* [twitch:stream] Expect 400 and 410 HTTP errors from API

* [tele5] Prefer jwplatform over nexx (closes ytdl-org#25533)

* [jwplatform] Add support for bypass geo restriction

* [tele5] Bypass geo restriction

* [ChangeLog] Actualize
[ci skip]

* release 2020.06.06

* [kaltura] Add support for multiple embeds on a webpage (closes ytdl-org#25523)

* [youtube] Extract chapters from JSON (closes ytdl-org#24819)

* [facebook] Support single-video ID links

I stumbled upon this at https://www.facebook.com/bwfbadminton/posts/10157127020046316 . No idea how prevalent it is yet.

* [youtube] Fix playlist and feed extraction (closes ytdl-org#25675)

* [youtube] Fix thumbnails extraction and remove uploader id extraction warning (closes ytdl-org#25676)

* [youtube] Fix upload date extraction

* [youtube] Improve view count extraction

* [youtube] Fix uploader id and uploader URL extraction

* [ChangeLog] Actualize
[ci skip]

* release 2020.06.16

* [youtube] Fix categories and improve tags extraction

* [youtube] Force old layout (closes ytdl-org#25682, closes ytdl-org#25683, closes ytdl-org#25680, closes ytdl-org#25686)

* [ChangeLog] Actualize
[ci skip]

* release 2020.06.16.1

* [brightcove] Improve embed detection (closes ytdl-org#25674)

* [bellmedia] add support for cp24.com clip URLs(closes ytdl-org#25764)

* [youtube:playlists] Extend _VALID_URL (closes ytdl-org#25810)

* [youtube] Prevent excess HTTP 301 (ytdl-org#25786)

* [wistia] Restrict embed regex (closes ytdl-org#25969)

* [youtube] Improve description extraction (closes ytdl-org#25937) (ytdl-org#25980)

* [youtube] Fix sigfunc name extraction (closes ytdl-org#26134, closes ytdl-org#26135, closes ytdl-org#26136, closes ytdl-org#26137)

* [ChangeLog] Actualize
[ci skip]

* release 2020.07.28

* [xhamster] Extend _VALID_URL (closes ytdl-org#25789) (ytdl-org#25804)

* [xhamster] Fix extraction (closes ytdl-org#26157) (ytdl-org#26254)

* [xhamster] Extend _VALID_URL (closes ytdl-org#25927)

Co-authored-by: Remita Amine <remitamine@gmail.com>
Co-authored-by: Sergey M․ <dstftw@gmail.com>
Co-authored-by: nmeum <soeren+github@soeren-tempel.net>
Co-authored-by: Roxedus <me@roxedus.dev>
Co-authored-by: Singwai Chan <c.singwai@gmail.com>
Co-authored-by: cdarlint <cdarlint@users.noreply.github.com>
Co-authored-by: Johannes N <31795504+jonolt@users.noreply.github.com>
Co-authored-by: jnozsc <jnozsc@gmail.com>
Co-authored-by: Moritz Patelscheck <moritz.patelscheck@campus.tu-berlin.de>
Co-authored-by: PB <3854688+uno20001@users.noreply.github.com>
Co-authored-by: Philipp Hagemeister <phihag@phihag.de>
Co-authored-by: Xaver Hellauer <software@hellauer.bayern>
Co-authored-by: d2au <d2au.dev@gmail.com>
Co-authored-by: Jan 'Yenda' Trmal <jtrmal@gmail.com>
Co-authored-by: jxu <7989982+jxu@users.noreply.github.com>
Co-authored-by: Martin Ström <name@my-domain.se>
Co-authored-by: The Hatsune Daishi <nao20010128@gmail.com>
Co-authored-by: tsia <github@tsia.de>
Co-authored-by: 3risian <59593325+3risian@users.noreply.github.com>
Co-authored-by: Tristan Waddington <tristan.waddington@gmail.com>
Co-authored-by: Devon Meunier <devon.meunier@gmail.com>
Co-authored-by: Felix Stupp <felix.stupp@outlook.com>
Co-authored-by: tom <tomster954@gmail.com>
Co-authored-by: AndrewMBL <62922222+AndrewMBL@users.noreply.github.com>
Co-authored-by: willbeaufoy <will@willbeaufoy.net>
Co-authored-by: Philipp Stehle <anderschwiedu@googlemail.com>
Co-authored-by: hh0rva1h <61889859+hh0rva1h@users.noreply.github.com>
Co-authored-by: comsomisha <shmelev1996@mail.ru>
Co-authored-by: TotalCaesar659 <14265316+TotalCaesar659@users.noreply.github.com>
Co-authored-by: Juan Francisco Cantero Hurtado <iam@juanfra.info>
Co-authored-by: Dave Loyall <dave@the-good-guys.net>
Co-authored-by: tlsssl <63866177+tlsssl@users.noreply.github.com>
Co-authored-by: Rob <ankenyr@gmail.com>
Co-authored-by: Michael Klein <github@a98shuttle.de>
Co-authored-by: JordanWeatherby <47519158+JordanWeatherby@users.noreply.github.com>
Co-authored-by: striker.sh <19488257+strikersh@users.noreply.github.com>
Co-authored-by: Matej Dujava <mdujava@gmail.com>
Co-authored-by: Glenn Slayden <5589855+glenn-slayden@users.noreply.github.com>
Co-authored-by: MRWITEK <mrvvitek@gmail.com>
Co-authored-by: JChris246 <43832407+JChris246@users.noreply.github.com>
Co-authored-by: TheRealDude2 <the.real.dude@gmx.de>
blackjack4494 referenced this issue in blackjack4494/youtube-dlc Sep 12, 2020
joedborg referenced this issue in joedborg/youtube-dl Nov 17, 2020
[pull] master from ytdl-org:master
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants