Please allow creation of internet shortcut files (.url, .webloc, .desktop)

[h-h-h-h]

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.05.01. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2018.05.01

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

---

Description of your issue, suggested solution and other information

What?

youtube-dl should be able to create internet shortcut files in different formats. That is, instead of downloading a file, a small textfile is created -- based on a template for the desired format -- that contains the respective URL.

[EDIT: Example file content:

[InternetShortcut]
URL=https://www.youtube.com/watch?v=4_jkeWgtZ18

]

Why?

This is useful, e.g., when you want to binge-watch YouTube channels and want to take care yourself of what you already watched and didn't watch, and in what order, instead of relying on the abilities of the video platform, but don't want to get your disk space reduced unnecessarily.

You can also drag .url files on Windows into a player like MPC-BE to stream them, which is much more performant than in the browser! Unlike with downloaded videos, it's still easy with an internet shortcut to get access to the video comments in the browser.

After binge-watching a channel, you can share a zip file with shortcut files of the best content with others.

How?

Each major OS has its own file format for internet shortcuts. Though on macOS, while there is no system-level support for the Windows format, many apps support it, if I understood that correctly:

• Windows: .url
• macOS: .webloc
• Linux: .desktop

Maybe, the --format switch could be extended with the identifier link or shortcut. This would default to the file format that is associated with the current platform.

To explicitly specify the file format, the following identifiers could be established: winlink, maclink, linuxlink.

Caveats for Windows: There are .url and .website files. "Based on exprimentation, I found programatically generating a .website file is non-trivial. If you don't get the parameters exactly right, it will not work properly. As of yet, I cannot find documentation describing the format. The name of the link appears to be embedded in the file." (source) So, we should stick to traditional .url shortcuts. (Firefox also creates .url files when dragging the address bar text.)

Caveats for Linux: .desktop files are described: "A Free Desktop Entry, used by Gnome and KDE, as well as Solaris (which I think uses Gnome)." (source) Therefore, I'm not sure, whether we can generalize .desktop files with linuxlink. (I'm not a Linux user.)

Alternatively or additionally, you may further specify the link identifier with the extension, or use it directly like --format mp4:

• --format link[ext=url]
• --format link[ext=webloc]
• --format link[ext=desktop]
• --format url
• --format webloc
• --format desktop

Proposed templates

The following placeholders are used in the templates that have to be replaced in a string-based manner, nothing fancy:

• {{{url}}} -- Different sections of the URL must be encoded differently. See https://news.ycombinator.com/item?id=11674220.
• {{{title}}} -- Only used for .desktop files. "Ubuntu...The file explorer displays the name embedded in the file." (source) Hence, it should probably be identical to the filename. Characters that are invalid in the file system should be okay according to these example files.

The file must be saved with UTF-8 encoding.

.url files

[InternetShortcut]\r\n
URL={{{url}}}\r\n

Chrome creates files of that structure.

Note: Windows internet shortcut files may also be created via COM. Here's a code example of MPC-BE for reading .url files via COM.

.webloc files

<?xml version="1.0" encoding="UTF-8"?>\n
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n
<plist version="1.0">\n
<dict>\n
	<key>URL</key>\n [\t for indent]
	<string>{{{url}}}</string>\n [\t for indent]
</dict>\n
</plist>\n

.desktop files

[Desktop Entry]\n
Encoding=UTF-8\n
Name={{{title}}}\n
Type=Link\n
URL={{{url}}}\n
Icon=text-html\n

Further information and things to consider

• There's a Web Shortcut Utility Suite, that just has libraries for Java and Perl. However, the developer writes:

During development I collected shortcuts from various operating systems (both to study and to use for unit tests). I also manually created some examples for testing purposes. I made these available in their own repository with the hope that they will be useful to other developers. [link]

• Windows is caching the URL of a .url file based on its filename or file path. This may mislead you when testing, or, possibly, a user when they run youtube-dl multiple times with simple output filenames.
• It's important, that the --download-archive switch for continuing watching a channel works with internet shortcut files, also, even when previously created shortcut files were deleted.

[dstftw]

This simply won't work: none of these shortcut formats apparently support passing HTTP headers. Needless to say it's extremely specific and out of scope.

[h-h-h-h]

@dstftw:

none of these shortcut formats apparently support passing HTTP headers.

What have HTTP headers got to do with this matter?

Imagine, I have a list of videos I want to watch, like: https://www.youtube.com/user/sxephil/videos. This list has videos with URLs of the form https://www.youtube.com/watch?v=4_jkeWgtZ18, which are the first thing youtube-dl encounters before trying to go deeper. These URLs must be written into the internet shortcut files, one of these URLs per file. MPC-BE, e.g., understands them. You can also double-click one of those and a browser tab will open.

Example .url file:

[InternetShortcut]
URL=https://www.youtube.com/watch?v=4_jkeWgtZ18

[EDIT: It's also useful for getting the URLs of an RSS feed without downloading the videos when you would rather like to stream them.]

I think, if more people had and knew these possibilities for binge-watching a channel and staying up-to-date with it or revisit the channel later for a further binge-watch session, the function would get its valid share of usage.

And what else is youtube-dl's goal but to enable people to get access to their videos under other circumstances than what the website provides for, even if it's "just" by acting as a means of checking what you have already watched (what you already have been presented via link files). (I'm referring to --download-archive.)

Needless to say it's extremely specific and out of scope.

Would you, please, alternatively, point me to the important locations in the codebase and tell me possible things to consider, so that I can try it myself -- at least off the top of your head?

1. Which route must I go to fetch the webpage URLs for single videos, based on a --format like link?
2. Where should the file writing take place?
3. Where/how must I change the parsing of the --format switch?
4. Where should I store the templates? (compiled-in strings [central location?], resource files)

[noureddin]

Interesting idea!

You can use youtube-dl --get-id with your channel/playlist url, and it will give you a list of video ids, each one on a line. For example, in Bash (GNU/Linux commandline):

youtube-dl --get-id $URL | while read id
do
	echo "[InternetShortcut]" > "$id.url"
	echo "URL=https://www.youtube.com/watch?v=$id" >> "$id.url"
done

This will create a file named ID.url with the contents you specified in your previous reply, for each line (or ID).

If you want to have the .url filename be the same as the video title, it's possible, but a little more complicated.

We need the --get-title switch. And then it will print the video titles and ids; each id or title is on a line by its own, and every title is before the corresponding id.

You can then use that output to generate these .url files. Using Bash (again), I can do something like this:

youtube-dl --get-title --get-id $URL | while read title
do
	read id
	echo "[InternetShortcut]" > "$title.url"
	echo "URL=https://www.youtube.com/watch?v=$id" >> "$title.url"
done

The problem is that using a video title as a filename directly is unsafe; you need to handle at least the forbidden characters in filenames in Windows (/, ?, <, >, \, :, *, |, ") by stripping them or replacing them by - for example.

[h-h-h-h]

@noureddin

Unfortunately, we also need to update a list of link files, and also find IDs of videos that were uploaded a while ago, but not made visible just until now, so that they appear inmidst of video IDs for which link files were already created. Further, I'd like the naming scheme ####. This is a video title.url, instead of yt_video_id.url that destroys the sort order, and still be able to ignore those already encountered files when doing updates in the future (even when the link files were deleted).

Using the ecosystem of youtube-dl with its --download-archive switch is much saner!

[rautamiekka]

@noureddin You need to keep in mind > empties the fiile and adds the texts whereas >> just adds the texts on new lines (if you target a file that doesn't exist it'll create it and add the texts starting from the first line). You've that mistake in the first, and possibly in the second script as well.

[noureddin]

Thanks, @rautamiekka!
Edited. :)

[noureddin]

@h-h-h-h

I just checked, and yes, you're correct; youtube-dl doesn't put anything in the download-archive file when simulating (e.g., using the --get-* switches), and I can't find anything in the help to make it do that.

About naming the files, the way I showed to create files using the video title can be easily changed to use the --get-filename switch instead of --get-title, and now, you have all the flexibility of the output templates.
The only thing is that an id is printed before the filename.

I think the only remaining problem is recording what videos are downloaded, even if their link files are deleted. This would be easy if there were a switch in youtube-dl to make it record simulated downloads in the archive file.

I agree that using youtube-dl is much saner. But if this functionality isn't going into youtube-dl, one can make a small Python script that uses youtube-dl to get all the needed metadata (in JSON maybe), read the archive file, write the link files, and append the newly “saved” videos to the archive file.
But I noticed that youtube-dl looks in the archive file, if given, even when simulating. This means we can handle recording the saved videos, and leave filtering old videos to youtube-dl itself.

My Bash prototype, again. :)

download_archive=.archive  # a hidden file
youtube-dl --get-id --get-filename --download-archive "$download_archive" \
		-o '%(playlist_index)s. %(title)s-%(id)s.%(ext)s' $URL |
while read id
do
	read filename
	filename="${filename%.*}.url"  # change the extension to .url
	echo "Downloading $id: $filename"  # be a little verbose
	echo '[InternetShortcut]' > "$filename.url"
	echo "URL=https://www.youtube.com/watch?v=$id" >> "$filename.url"
	echo "youtube $id" >> "$download_archive"  # record in the archive,
	# in the same format youtube-dl recognizes
done

[h-h-h-h]

@dstftw, @phihag, @jaimeMF, @yan12125, @remitamine, @kennell

Please re-evaluate this issue.

[h-h-h-h]

@noureddin:

I noticed that youtube-dl looks in the archive file, if given, even when simulating. This means we can handle recording the saved videos, and leave filtering old videos to youtube-dl itself.

Thanks, that's important information, should we need to write a solution on top of youtube-dl.

[h-h-h-h]

I created a pull request that implements this feature: #16455.