Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Extended Date/Time Output Templating Options #13752

Closed
wants to merge 56 commits into from
Closed

[Enhancement] Extended Date/Time Output Templating Options #13752

wants to merge 56 commits into from

Conversation

kade-robertson
Copy link

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

The purpose of the pull request is to add a more flexible output for organizing very specifically by the date and/or time of the upload. The string formatting available is fairly limited, so numerous extensions were added to include the option of using the:

  • year
  • month (numeric and localized string, long or short form)
  • day (numeric and localized string, long or short form)
  • hour
  • minute
  • second
  • AM/PM indicator

The latter four do require a timestamp value to be present to provide useful data, however they will not cause errors if only the upload date is present (they will just be their defaults). Two extra output options were added for the full time string in 24 or 12-hour format. In the event that neither are present these values are simply ignored so this should be future-proof as long as upload_date's format doesn't change, and if it does it's a fix on a single line. The names were chosen to try and be as descriptive as possible while keeping short enough to fit reasonably in a line, and to not overlap as much as possible.

The desire for such a change was for easy archiving of many channels across long timespans, which is now possible if, for example, a template like this is used:

%(uploader)s/%(upload_year)s/%(upload_month)s-%(upload_mntn)s/%(upload_day)s/%(upload_time24)s-%(title)s-%(id)s.%(ext)s

@dstftw
Copy link
Collaborator

dstftw commented Jul 27, 2017

Instead of introducing enormous number of new meta fields upload_date and timestamp (as well as all other date meta fields) should support strftime-like syntax in output template.

@kade-robertson
Copy link
Author

How would you suggest this be implemented with the current string formatting procedure? Some sort of pre-processing of the template?

@dstftw
Copy link
Collaborator

dstftw commented Jul 27, 2017

There is no way to do this without preprocessing.

@kade-robertson
Copy link
Author

Hm, I'm not really sure of what an ideal clean solution would look like, what if there was an exclamation delimiter like upload_date!%A which indicated the string should be pre-processed to be strftime format. I don't really know the best solution for this because I would imagine it should also comply afterwards with the user's formatting, i.e. if they just want the first four letters of the output they should still be able to do %(upload_date!%H%M%s).4s right?

It would be relatively simple to use a regex replacement sort of setup but it wouldn't be compatible with the above, something like r'%\((upload_date|timestamp)!(%.*)+?\)s'

@dstftw
Copy link
Collaborator

dstftw commented Jul 29, 2017

The general idea is to allow something like %(field[:<field_specific_formatter>])<standard_formatter>.

dstftw and others added 25 commits February 11, 2018 09:34
(also fixed test broken by changed og tags on the site)
they seem to be required now, else we get 403 errors.
* Revert "[francetv] Separate main extractor and rework others to delegate to it"

This reverts commit 24b97ef.

* Revert "[francetv] Improve manifest URL signing (closes #15536)"

This reverts commit 6f54dce.

* Revert "[francetv] Sign m3u8 manifest URLs (closes #15565)"

This reverts commit 4eb4ace.

* Revert "[veoh] Add support for embed URLs"

This reverts commit 6adfc88.

* Revert "[dvtv] Skip download on failing test"

This reverts commit ad4167e.

* Revert "[afreecatv] Fix extraction (closes #15556)"

This reverts commit 8cc0cd0.

* Revert "[periscope] Use accessVideoPublic endpoint (closes #15554)"

This reverts commit b6aef8f.

* Revert "[YoutubeDL] Add support for filesize_approx in format selector (closes #15550)"

This reverts commit eb048b3.

* Revert "[discovery] Fix auth request (closes #15542)"

This reverts commit 14adea5.

* Revert "[6play] Extract subtitles (closes #15541)"

This reverts commit aee4411.

* Revert "Credit @mweinelt for #15124"

This reverts commit 82eaab7.

* Revert "Credit @iamleot for internazionale (#14973)"

This reverts commit fdd87ce.

* Revert "Credit @che0 for seznamzpravy (#14616) and dvtv (#15442)"

This reverts commit 9b904aa.

* Revert "[newgrounds] Fix metadata extraction (closes #15531)"

This reverts commit 8be35d3.

* Revert "[nbc] add support for NBC Olympics Streams(closes #10295)"

This reverts commit ba6e116.

* Revert "[dvtv] Simplify (closes #15442)"

This reverts commit 2da0581.

* Revert "[dvtv] Fix live streams extraction"

This reverts commit 7732346.

* Revert "release 2018.02.08"

This reverts commit 0204b2a.

* Revert "[ChangeLog] Actualize"

This reverts commit 72aa0a2.

* Revert "[extractors] Import for myvi:embed"

This reverts commit 38e519f.

* Revert "[pokemon] PEP 8"

This reverts commit 7b11f92.

* Revert "[gameinformer] PEP 8"

This reverts commit aa71431.

* Revert "[myvi] Extend _VALID_URL"

This reverts commit 441ec65.

* Revert "[myvi:embed] Add extractor (closes #15521)"

This reverts commit e35a984.

* Revert "[prosiebensat1] Extend _VALID_URL (closes #15520)"

This reverts commit 0e1a732.

* Revert "[pokemon] Relax _VALID_URL and extend title extraction (closes #15518)"

This reverts commit 8a768fe.

* Revert "[gameinformer] Use geo verification headers"

This reverts commit 6cdc00e.

* Revert "[la7] Fix extraction (closes #15501)"

This reverts commit 8d2fbb0.

* Revert "[gameinformer] Fix brightcove id extraction"

This reverts commit dc1171d.

* Revert "[afreecatv] Pass referrer to video info request (closes #15507)"

This reverts commit c16af2c.

* Revert "[telebruxelles] Relax _VALID_URL and add support for live streams"

This reverts commit 1f916ea.

* Revert "[telebruxelles] Fix extraction (closes #15504)"

This reverts commit ea7a8f5.

* Revert "[extractor/common] Respect secure schemes in _extract_wowza_formats"

This reverts commit b44f7f4.

* Revert "release 2018.02.04"

This reverts commit ed4de10.

* Revert "[ChangeLog] Actualize"

This reverts commit f277eb6.

* Revert "[brightcove] Pass embed page URL as referrer (closes #15486)"

This reverts commit a690cb6.

* Revert "[downloader/http] Randomize HTTP chunk size"

This reverts commit c315342.

* Revert "[youtube] Enforce using chunked HTTP downloading for DASH formats"

This reverts commit 7412fe8.

* Revert "[downloader/http] Add ability to pass downloader options via info dict"

This reverts commit 6cfd25a.

* Revert "[downloader/http] Fix 302 infinite loops by not reusing requests"

This reverts commit 57c3eea.

* Revert "Document http_chunk_size"

This reverts commit ff3f520.
@dstftw dstftw closed this Feb 11, 2018
@cferrey
Copy link

cferrey commented Jan 8, 2019

Apologies if I'm violating a rule by posting to this thread, but could someone help me with the syntax for calling this feature? I've tried every iteration I can think of, but keep getting "NA" or "ERROR: not enough arguments for format string" depending on where I try to put the formatting arguments.

I am hoping to call %(upload_date)s three times for %Y, %M, %D and delimit each output with a period, in order to turn 20190108 into 2019.01.08

Thanks!

@dstftw
Copy link
Collaborator

dstftw commented Jan 8, 2019

No such feature exists.

@cferrey
Copy link

cferrey commented Jan 8, 2019

Got it -- thank you for the info. Are there any plans to implement the above? If not, is there any other way I can accomplish this with youtube-dl alone, or do I need to rename after the fact?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants