[Spec] Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL

[achimmihca]

Suggestion

Melody Mania uses the #WEBSITE header field to specify a website that should be used in its embedded browser.

This way, it is possible to specify a youtube link for example that can be used as alternative to local audio and video files.

Note that the same approach might also work with other platforms such as Spotify and Apple Music.
Thus, I suggest to use WEBSITE instead of platform specific names such as YOUTUBE or similar.

Note further that WEBSITE fits with other header names of the format, e.g. VIDEO, BACKGROUND, and COVER. These header fields describe what the header value points to.

Use case

• Link to or embed the website for a preview of a fitting audio / video for the txt file (i.e. with matching GAP, VIDEOGAP, etc.)
• Use website as alternative to local audio and video files (e.g. in Melody Mania)

Extra info/examples/attachments

No response

[Baklap4]

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ?
If yes this can potentially lead to security/malware risks if no whitelist is made
Personally i'm more fan of using an enum/typed field since this is self-explainatory, easy to implement and not prone to beforementioned risks

[achimmihca]

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ?

Yes.

this can potentially lead to security/malware risks if no whitelist is made

I would consider this an implementation detail of karaoke players, not part of the spec.

For instance, Melody Mania loads only websites from supported hosts. And a new host can be supported by adding a corresponding JavaScript file.
So you have a whitelist mechanism there. But other players may use a different approach.

As I said, I would not consider whitelisting part of the spec.

i'm more fan of using an enum/typed field

New platforms may arrive in the future. Further, communities might want to host their own song server.
Thus, I don't think that an enum with a list of supported platforms is sufficient.

Which websites can be used should be a config issue of the player in one way or another. Not part of the spec.

[Baklap4]

How would this be combined with say for example our current #MP3 header or the newly introduced #AUDIO or in the later future maybe both #AUDIO and #INSTRUMENTAL? Would this field contain multiple values seperated by character or would we have multiple #WEBSITE tags

If we're going to introduce this tag it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file.

Now thinking of it... what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields?
If no http is found search local. If the string is found load from web? This way we don't need an extra field

Within performous we can for example specify a NAS through the network share \\NAS\Somefolder and it'll load everything just fine

[achimmihca]

Good question.
My goal is to provide a fallback mechanism when local files are not present.

it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file

Yes.

what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields

Nothing. Actually, Melody Mania and UltraStar Play accept an http URL in these fields that points to an audio / video / image file.

However, my goal is to provide a fallback mechanism when local files are not present. Thereby, it would be nice to stay compatible with offline play in existing UltraStar games.

Thus, I argue that a new field is required for this.

---

But you are right, other header fields could make use of an online fallback as well.
So maybe the field should accept a key value list, where the keys are other header fields.

Further, I would expect the fallback to be something online (e.g., a URI, YouTube video ID, or some other identifier).
So maybe a better name for the field would be #ONLINE or #CLOUD?

Example:

#ONLINE: audio="https://something.com/audio.ogg",cover="https://something.com/image.png"

I think this would be similar to what people are doing with new txt files on usdb.animux.de.
Their goal is also to make the txt file self-sufficient such that other stuff can be fetched when needed. At the moment they use the #VIDEO field for this, which breaks compatibility with local video files. A standardized approach for this would be nice.

[marwin89]

SUMMARY

So here is a summary for this issue up till now.

We add this tag to version 1.1.0:

#ONLINE: audio=1k8craCGpgs, video=yCC_b5WHLX0, cover=heart-skips-a-beat-57c3df3b913b0.jpg, background=4f9954bd84f67.jpg

The description in the spec (website) could be:

#ONLINE
This tag links to different online ressources and webservices. You can link for a preview/reference/embed of a fitting audio or video that matches with GAP, VIDEOGAP, in the txt-file. Use IDs or URLs from the webservices of your choice (for example: Youtube, Spotify, Apple Music, Vimeo etc.). Right now the tag can only be 255 characters long.
Allowed keys are: audio=, video=, cover=, background=
Allowed values are: ID (prefered), complete url

@bohning : could you help us out here with usdb syncer meta-tags ? That's a lot to document here. You also got advanced features: cropping, rotating and resizing of covers.

[Nianna]

Can't we use several flat tags for this?
e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have
#KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

[Baklap4]

I can see why it's been done for USDB_Syncer and the #VIDEO tag as the syncer processes it and makes a #KEY:VALUE header which is easy to parse (which i think is good). Introducing multi-key value within the text file without processing after can make things a little bit tricker and more complex

[bohning]

Can't we use several flat tags for this? e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have #KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

Then I would vote for #COVERURL, #VIDEOURLand #AUDIOURL (and sort them right after the respective offline tags), so that

#AUDIO:xyz.mp3
#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEO:xyz.mp4
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVER:xyz [CO].jpg
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUND:xyz [BG].jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

The main reason for using YT and fanart.tv IDs for usdb/usdb syncer is to save characters (max. line length = 255 chars) and to push people towards using those two platforms over others. If the source URLs get their own tag, then at least the first reason vanishes.

[achimmihca]

keeping a flat, easy-to-understand structure

I agree

• the first suggestion (maybe name it nested-key-value) would add a second key=value mechanism to the file format, which is redundant
  • if we decide to use this then syntax should be symmetric with [Spec] Add tags for online music databases #19
    • BTW: in this scenario, should we separate mulitple keys-value pairs with comma or ampersand? I suggest to use ampersand such that existing URL-parsing libs can be reused.
• @bohning's suggestion (maybe name it flat-key-value) also works for me
  • on first glance, it is simpler and more intuitive

---

In any case, I would prefer that we keep the file format consistent.
In the flat-key-value scenario, should we also prefer a flat-key-value style for #19 ? For example

#MUSICDBX:VALUEX
#MUSICDBY:VALUEY
...

[Nianna]

should we also prefer a flat-key-value style for #19

Yes, let's use flat structure for all tags.

[bohning]

We might want/need to add

#COVERROTATE:[angle in degrees ccw -> float]
#COVERCROP:[left-upper-width-height]
#COVERRESIZE:[width --> square: width=height]

if online resources are used. Similarly, for backgrounds (but not rotation needed):

#BACKGROUNDCROP:
#BACKGROUNDRESIZE:

These are features that the syncer uses to provide square covers and e.g. 16:9 backgrounds (furthermore, it has the option to increase the image contrast for covers --> #COVERCONTRAST:).

[marwin89]

2nd SUMMARY / VOTE NOW 📈 ✋

Thanks for the discussion. So here is the second summary for this issue. We are close to a final result.

Please vote for this issue till 15th nov 2023 with emoticon 🎉 / 🚀.

Options

🚀 A: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and Standard crop/resizing method in USDB SYNCER with central clipping

🎉 B: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and #COVERROTATE, #COVERCROP, #COVERRESIZE, #BACKGROUNDCROP, #BACKGROUNDSIZE

👀 C: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL
and tags are not flat anymore

❤️ D: Change value structure in #AUDIO/#MP3, #COVER, #VIDEO, #BACKGROUND from flat to complex (for txt-file hostings).

What?

• Adding # attributes to identify online files vom webservices that fits to txt-file.
• making txtfiles self sufficent and/or "streamable".
• Song Editors just have to enable editing the # attributes
• All downloading functionality is done by USDB SYNCER (and optionally by games)
• Games can use it for fallback or streaming, but don't have to

Why?

• USDB-SYNCER and USDB: the URLs/IDs help usdb-syncer to know the sources to load.
• FOR SONG REWORKS: It helps Song Creators to understand on which file basis a song was originally made at all, there are a lot of messy txt files in usdb that are build-on obscure mp3 with wrong bpm that nobody understands anymore. with audio/videourl this problems are history.
• PREVIEW/FALLBACK of missing local files in game and streamable alternative to local files (initial suggestion of achimmihca in this issue,)

---

details A 🚀:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

These tags are optional.

➕ Built-in standard method in USDB SYNCER that crops/resizes cover and background images with centered clipping to max Full-HD. Many CD Covers will not be cropped good.

---

details B 🎉:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#COVERROTATE:-0.3
#COVERCROP:0-200-1000-1000
#COVERRESIZE:1920
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDCROP:10-10-1920-1080
#BACKGROUNDRESIZE:1920-1080

These tags are optional.

details C 👀 :

Introduce complex tags

#AUDIOURL:https=asdfghjkloi
#VIDEOURL:https=asdfghjkloi
#COVERURL:https=asdfghjkloi,resize=500x500,crop=0-200-1000-1000,rotate=-0.1
#BACKGROUNDURL:https=asdfghjkloi,resize=500x500,crop=10-10-1920-1080

details D ❤️ :

We change the value structure in existing # Attributes in the next version v.1.1.0 from flat to complex.
(Only if usdb.animux gets an improvement from Alex! Or new USDB.) In the TXT-FILE Hostings the value structure would be

#AUDIO/#MP3:url
#COVER:url,rotate=,crop=,resize=
#BACKGROUND:url,resize=,crop=
#VIDEO:url,trim=,crop=

#Audio/#MP3 stays mandatory, the others stay optional. Games detect files in the specific song folder by file type for instance .mp3, .wav .ogg = Audio, mp4 m4a wmv = VIDEO, Cover = jpg / png file name contains [CO], Background = jpg png file name contains [BG].

---

The description in the spec (website) could be:

#AUDIOURL
This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting audio that matches with GAP, VIDEOGAP, in the txt-file. Use complete URLs from the webservices of your choice (for example: Youtube, Vimeo etc.). Right now the tag can only be 255 characters long.
#VIDEOURL
This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting video that matches with GAP, VIDEOGAP, in the txt-file. Use complete URLs from the webservices of your choice (for example: Youtube, Vimeo etc.). Right now the tag can only be 255 characters long.
#COVERURL
This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting album art for the song. Use complete URLs from the webservices of your choice (we recommend fanartTV). Right now the tag can only be 255 characters long.
#BACKGROUNDURL
This tag links to an online background image file that fits the song. Use complete URLs from the webservices of your choice (we recommend fanartTV). Right now the tag can only be 255 characters long.
#COVERROTATE:
Rotates the cover-image from #COVERURL. [angle in degrees ccw -> float]. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.
#COVERCROP:
Crops the cover-image from #COVERURL. [left-upper-width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.
#COVERRESIZE:
Resizes the cover-image form #COVERURL. [width --> square: width=height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.
#BACKGROUNDCROP:
Crops the background image from BACKGROUNDUR [left-upper-width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.
#BACKGROUNDRESIZE:
Resizes the background image from BACKGROUNDURL [width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

[rakuri255]

I do not understand the purpose of these tags.
Why should we add a url, which often has no long life? When I look at my Youtube playlist alone, which I created last month, 2 out of 30 videos have already been deleted.
These are all tags that can be deprecated in a few months anyway.
In my opinion, the txt file should contain all the information offline.

Looks to me like overengineering.

[marwin89]

@rakuri255 sorry, my summary was missing some explanation.

• the URLs/IDs are for the usdb-syncer, to know the sources.
• They fit to melody manias usecase of preview/fallback of missing files too (initial suggestion of achimmihca in this issue)
• two birds with one stone

[basisbit]

I agree with @rakuri255, imho this will not pass the test of time, but instead just cause frustration / confusion for users and unnecessary complexity for devs. If the tags are only for usdb syncer, then you can just as well not have them in the standard. All major implementations of the format support "unknown" tags by ignoring them but not removing them when saving any changes to the file.

[achimmihca]

If the tags are only for usdb syncer, then you can just as well not have them in the standard

I agree. There is no point in adding something to the spec when it is only used by a single tool.

Futher, crop / resize / rotate: this could be specified as part of a BACKGROUNDURL (e.g. as query parameters), so I don't think a dedicated header field is required here.

That said, I still think that a common approach to load stuff from the cloud when not yet present locally would be beneficial to the format.

---

Please vote for this issue till next saturday

As others pointed out, most people contribute to these projects only occasionally, so a week is probably too short for voting. At least a month feels more appropriate to me.

[marwin89]

1. Please, IMO think and discuss more constructive especially about this feature.
2. Instead of saying no, say how you could imagine how this might work - because I believe you have good ideas and behind that request is a certain need
3. As a user I always wondered (years ago) where I could store the info about the video I found on vimeo, youtube etc. that fits to the song (gap). as a newcomer I would simply open txt and paste urls in. This is a fact we should use.
4. in the end this is good for making the txt-files self-sufficient (and independed from one txt-hosting-databases), even if the games don't use it in game play.
5. these tags (URLS/IDs) are used in usdb-syncer, usdb, the editors - its a general thing
6. of course youtube links are taken down, but with the work of community in usdb and the use of usdb-syncer they get updated well.
7. Maybe we can use query parameters in the urls - to reduce the #tag amount. Let's ask bohning.
8. we miss a big point if we don't include that in the standard. This feature should be official.

Futher, crop / resize / rotate: this could be specified as part of a BACKGROUNDURL (e.g. as query parameters), so I don't think a dedicated header field is required here.

@bohning, is that possible with query paramaters? what do you think about that?

[bohning]

I will think about it and get back to this.

[bohning]

Well, not so easy to sort my thoughts, but I’ll try to give it a shot. I really much like both the idea of a self-contained text file (or, what I think @basisbit recommended in a different issue: self-contained song zip file) as well as the idea of dynamically loading resources, either as a secondary, fallback source or even to avoid having local files all together.

One big problem of usdb was always that most content creators did not bother commenting the audio/video used and that’s where everybody else started getting busy, trying different versions, adapting GAP and BPM, not seldomly without success because the creator used some obscure version and failed to mention that.

With the USDB Syncer and metatags in place, this is a thing of the past and if resources (audio, video, cover, background) get outdated, this can be fixed quite quickly with an active community. So I agree with Marwin that this info should be included in the txt.

If we don’t want to introduce the new tags #AUDIOURL etc., we could go for stating offline and online source in a single tag, e.g.
#AUDIO:Journey - Don’t Stop Believin’.m4a|https://www.youtube.com/watch?v=1k8craCGpgs
#COVER:Journey - Don’t Stop Believin’ [CO].jpg|https://images.fanart.tv/fanart/dont-stop-believin-5e455e58e7b13.jpg
etc.

That would enable the syncer to download the resources, but also users that do not use the syncer would know the used/recommended resources and karaoke software could be adapted to use either the local or online resources.

Apart from the facts that 1. usdb currently does not support this (#MP3, #COVER, #BACKGROUND are either deleted or autogenerated and overwritten from #ARTIST/#TITLE values) and 2. no karaoke software supports remote resources (UltraStar Play/Melody Mania does for remote files, but not remote video portal URLs such as a YT URL), this could be a way to go.

[DoubleDee73]

Personally, I don't think it's a good idea to add functionality to the existing tags. I think, it'd be cleaner to have separate tags, that the clients may chose to ignore. I think there's a character limit per tag, so sharing information may be problematic.
I also think, that additional tags like crop size, rotation, etc. are over-engineered. These could indeed be added to newly added tags, like
#BACKGROUND-URL:https:bla|resize:500x500|rotate:bla

From a dev point of view for Yass, I'm not sure, how to deal with this anyway. Just enabling editing, or also enable downloads. People would have to have additional libraries and stuff for decoding YT, or I would have to bundle it. It'd be more complex in any case...

[marwin89]

From a dev point of view for Yass, I'm not sure, how to deal with this anyway. Just enabling editing, or also enable downloads.

@DoubleDee73, @Nianna, @rakuri255 don't worry.
For EDITOR DEVs it's just enabling editing the #... attributes. ✍️
All download functionallity is done by USDB SYNCER. ⬇️

I also think, that additional tags like crop size, rotation, etc. are over-engineered.

I agree. Yet I see the need behind that. So I suggest following @bohning :

1. Do not use # attributes for finetuning Co/Bg,
2. but use a build-in automatic standard cropping/resizing method for covers and backgrounds in USDB SYNCER.
3. Why? Cause I presume almost all of the cases are done at least good with one method. 1️⃣all
4. Standard method for covers: cropping and resizing to 1:1 (max 1920px) with centered clipping. ✂️
5. Standard method for backgrounds: cropping and resizing 16:9 (max Full HD) with centered clipping. ✂️
6. leave out cover-rotating ❌

#BACKGROUND-URL:https:bla|resize:500x500|rotate:bla

Good idea :) I know we had the FLATKEY discussion already, yet I think this feature is so helpful, that we can skip Flatness for that.

I'll update the voting summary (considering your and bohnings thoughts).

[codello]

What is the state of this proposal? If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue. This could be similar to how HTTP handles headers:

• Single-Valued headers/tags can only be specified once. If a header is specified multiple times it is implementation dependent which value takes precedence (or we define it to be the first/last one).
• Multi-Valued headers can have multiple values separated by a comma. This is the consensus that was reached in [Spec] Multiple entries in #GENRE / #LANGUAGE / #CREATOR #30 for GENRE, LANGUAGE, etc.
• Multi-Valued headers can also be specified multiple times. Multiple occurrences are then concatenated with a comma. This solves the issue about a max line length.

(The same behavior should of course be applied to other multi-valued tags)

I think this would be a quite elegant solution for this proposal: The fields AUDIO, VIDEO, BACKGROUND, and COVER become multi-valued. The semantics of multiple values in a single field are that different values are to be tried in order. So a header like this:

#AUDIO:foobar.mp3
#AUDIO:file://nas.local/foobar.mp3
#AUDIO:https://youtube.com/…

Would semantically be equivalent to:

#AUDIO:foobar.mp3,file://nas.local/foobar.mp3,https://youtube.com/…

When loading such a song a program would first try to load the local foobar.mp3, then try the file located on a network share and as a final fallback try the YouTube URL (if supported by the client). Alternatively the spec could allow clients to choose a file by their own preference (which would be my preferred solution).

From a specification perspective each value would be a URL. If a value does not include a URL scheme it‘s understood to be a local file located relative to the txt file.

One drawback of this solution is that file names that contain a comma could not be used. I think this is an acceptable limitation but could be solved by implementing an escape sequence of some sort.

[DoubleDee73]

@codello For now, my implementation is single-valued only, but that doesn't mean, it couldn't be multi-valued. To be honest, personally, I'm not sure, what good it would do, to have multi-values anyway. The games would have to have some sort of fallback strategy, if a file was not found locally, to look into a NAS, or on youtube. The path to the NAS would be very user-specific anyway. I doubt, that would have a shared value.

[achimmihca]

If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue

We had the discussion of multi-value vs. single-value (aka. nested vs. flat) tags before.
We decided to stay with single-value (flat) tags. For details, see above comments.

[codello]

I think I may have worded that badly.

I read the whole discussion in this issue and I understand the nested vs. flat argument. On that I don't have a strong opinion either way but from what I gathered there seemed to be a tendency towards simpler structures (so flat tags).

To me the multi-value vs. single-value is a different point. The spec already includes multi-valued tags (see #30). I was thinking that maybe the same mechanism could be used to address the need for URL-based files. I'm not suggesting to introduce a complex sub-language for the media files. I just thought that maybe the existing multi-value system might me a good fit as it would alleviate the need for additional tags.

I'll try and phrase my suggestion a bit differently:

---

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

...
AUDIO:foobar.mp3
AUDIO:https://youtube.com/...
...

The exact semantics of this would have to be decided. I can think of two plausible approaches:

• The order of precedence is the order in which the tags appear in the file. So in the example above the local file foobar.mp3 would be the preferred audio source and if it does not exist or cannot be read, the YouTube link serves as a fallback.
• The order of precedence is an implementation detail and thereby defined by the application. I think I'd actually prefer this solution as it would give the end users as opposed to the song creators the choice of preferred media source.

In addition to this I'd like to unify this syntax of repeating a tag with the existing multi-valued tags in order to reduce the complexity of the spec. But this aspect can be discussed separately.

[Baklap4]

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

AUDIO:foobar.mp3
AUDIO:https://youtube.com/...

The web usually handles multiple same keys in this way: ?audio=foobar.mp3&audio=youtube.com/...

Basicly the same as @codello is suggesting. I'd choose @codello answere over any other mentioned solution

[achimmihca]

I like your suggestion in general.
However, I am unsure whether this is compatible with old implementations.

The solution with new tags has the advantage that it is backwards compatible because unknown tags are ignored.

[codello]

That's a good point. I don't think it's fundamentally incompatible, but in practice it would be implementation-dependent. There is currently no spec that mandates that headers/tags cannot occur multiple times but I think most implementations assume that's the case.

Specifically USDX should actually be compatible with this suggestion. USDX issues an Error log entry for every MP3 (or VIDEO etc.) entry that does not resolve to a file but if at least one MP3 value actually resolves to an existing file, the song will still load without problems. If multiple MP3 values resolve to files, the last one will be used by USDX.

I don't know how other implementations would deal with repeated headers, but the most straightforward handling would be to just use the last value. So if an implementation doesn't specifically forbid the use of repeated headers this should be backwards compatible.

If we combine this with the multi-valued header syntax used for #TAGS etc. then that part would not be backwards-compatible. So maybe it would be best to defer that part to v2 of the spec which contains breaking changes anyways (or defer even the repeated headers to v2, just to be safe).