[Feat]: Upgrade Music Video from YouTube to Tidal

[Hendo16]

I know this functionality doesn't exist just yet but if we provide an additional parameter to allow for a video upgrade, it'd be great if there was a way the application could, instead of skipping artists it's already downloading, check all of the songs again in MusicBrainz to see if a tidal video link. Maybe when songs are downloaded an attribute is added to the nfo to flag if it was from youtube or tidal - that way as the database grows and higher quality videos are linked it'll get upgraded when the script is run.

[RandomNinjaAtk]

This is a good idea, I'm just not sure if anyone is actually populating the database with links... Would be nice if they did...

The script already prefers tidal when possible.

[Hendo16]

Well since finding this service, i've been working on the MusicBrainz DB, made over 1000 edits which include adding music videos so that this script will detect it, and adding Tidal video links where possible. But it's been getting frustrating since sometimes the cached MusicBrainz DB won't detect that it needs to update so i'll need to manually delete the cache alongside the artist 'completed' file for the service to actually download the latest video, and there's been multiple instances where it will pull a youtube link from IMVDB and download that song anyway, meaning there's two identical videos yet ones of a higher quality since it was pulled from Tidal. It'd be good if we could bypass IMVDB and only use that as a backup since MusicBrainz seems more flexable and easier for general users to update

[RandomNinjaAtk]

I would prefer to use musicbrainz as the only source, but the lack of videos linked, meant that IMVDB was necessary. Because the script is using both sources, the only valid way to really check for duplicates, is by file name, and that is not ideal because if title names or simple capitalization differences can cause a duplicate... Its not an easy problem to solve... Using a single database is much simpler to solve the issue but that also requires the community to step up and take action, which it appears that you've been doing so, so great job working on that.

Right now the script waits 7 days before re-checking musicbrainz for cache differences, see here:

docker-lidarr-extended/root/scripts/Video.sh

Lines 119 to 124 in aa5c68b

	if [ -f "/config/extended/cache/musicbrainz/$lidarrArtistId--$lidarrArtistMusicbrainzId--recordings.json" ]; then
	if ! [[ $(find "/config/extended/cache/musicbrainz" -type f -name "$lidarrArtistId--$lidarrArtistMusicbrainzId--recordings.json" -mtime +7 -print) ]]; then
	log "$processCount of $lidarrArtistIdsCount :: MBZDB :: $lidarrArtistName :: Previously cached, skipping..."
	return
	fi
	fi

After a 7 day timeout, it will then compare the cached number of recordings vs the online db number of recordings:

docker-lidarr-extended/root/scripts/Video.sh

Lines 126 to 132 in aa5c68b

	if [ -f "/config/extended/cache/musicbrainz/$lidarrArtistId--$lidarrArtistMusicbrainzId--recordings.json" ]; then
	musicbrainzArtistDownloadedRecordingsCount=$(cat "/config/extended/cache/musicbrainz/$lidarrArtistId--$lidarrArtistMusicbrainzId--recordings.json" | jq -r .id | wc -l)
	if [ $musicbrainzArtistRecordingsCount -ne $musicbrainzArtistDownloadedRecordingsCount ]; then
	log "$processCount of $lidarrArtistIdsCount :: MBZDB :: $lidarrArtistName :: Previously cached, data needs to be updated..."
	rm "/config/extended/cache/musicbrainz/$lidarrArtistId--$lidarrArtistMusicbrainzId--recordings.json"
	fi
	fi

If the Musicbrainz online db has more recordings, then the local cache is updated. So it will not detect changes, if the recording account stays the same, and you only add a url to an existing recording.... Don't think there is a better way to handle it honestly....

So here are some ideas I have that could potentially improve things based on the discussion here:

1. Make IMVDB or Musicbrainz optional, there would be no fallback mechanism
2. Remove the 7 day cache timeout for musicbrainz, forcing it to check for new recordings everytime the script runs. (should likely reduce the need for manual cache wipes.)
3. When the local cache receives an update, remove the artist completed file to allow it to re-process the artist automatically.
4. Add a source tag to the nfo's to attempt to detect and upgrade files (adding this feature would likely cause every existing file to be re-downloaded...)

Let me know your thoughts.

[RandomNinjaAtk]

2 & 3 have been implemented, but have not been tested....

[RandomNinjaAtk]

2, 3 & 4 should now be implemented.

[RandomNinjaAtk]

The dupe detection change seems to possibly eliminate the need for 1. Give the latest updates a try and see what you think... I think this is fully implemented now....