Import meta/tags using provided URL while download new archives by built-in downloader

[xiazeyu]

When want to download an archive from the Internet,
I use the Adding Archives to the Library function combined with Metadata Plugins, hoping to automatically add meta.

Take E-Hentai plugin for example, It seems that if I turn on "Run Automatically" and give "URL(s) to download".

It will always match metadata using TITLE instead of URL I provided in the downloader.

This can result in mismatch when the title contains Unicode characters, or multiple gallary shares the similar title.

My current solution is to disable "Run Automatically" option and Use "Click here to edit metadata." function to manually paste the "E-H Gallery URL (Will attach tags matching this exact gallery to your archive)".

Is it possible to just let "Mayriad's EH Master Script" to retrieve the URL I provided in the downloader, and use that information to search for metadata?

[Difegue]

That should indeed be the expected behavior -- I'm pretty sure the metadata plugins will match on URL as long as said URL is correct.

There's been some fixes in nightlies recently (#593) to be more permissive on source parsing, that might help!

[xiazeyu]

I looked into the Meta plugin, found

LANraragi/lib/LANraragi/Plugin/Metadata/MEMS.pm

Line 64 in 19756a1

} elsif ( $lrr_info->{existing_tags} =~ /source:e(?:x|-)hentai\.org\/g\/(\d+)\/([0-9a-z]+)/i ) {

LANraragi/lib/LANraragi/Plugin/Metadata/EHentai.pm

Line 72 in 19756a1

} elsif ( $lrr_info->{existing_tags} =~ /.*source:\s*e(?:x|-)hentai\.org\/g\/([0-9]*)\/([0-z]*)\/*.*/gi ) {

these codes seems to extract the "source" tag from existing tags.

However, when I download a new comic using E*Hentai Downloader v.1.0 by Difegue, the only tag I seen is "date_added:1649757812". These's no tag like "source" added, even though I download it by "Add Archives". Is it expected? Or should the "source" tag being added after the file is downloaded(just like how the "date_added" tag is added), and before Meta plugin is called?

[xiazeyu]

Another question, may be off-topic. Is there any difference between E-Hentai v.2.5.1 by Difegue and others and Mayriad's EH Master Script v.1.1.0 by Mayriad?
It seems that the MEMS just "[Mayriad's EH Master Script] [info] Source identified. Calling E-Hentai metadata plugin to retrieve metadata from EH API."

[Difegue]

The MEMS plugin is only useful if you have the matching userscript on your browser: The base EH plugin does everything you'd need as a regular user.

As for downloading through add archives, it calls on the DL system all the same so you should be getting source tags.
More investigating needed..

[xiazeyu]

It is confirmed that if I download other books without using the plugins:
e.g. https://github.com/sarabander/sicp-pdf/raw/master/sicp.pdf

source tag is not added. I think it is not what I expected, correct?

Related code found here, still investigating.

LANraragi/lib/LANraragi/Utils/Minion.pm

Lines 219 to 223 in 6c1b241

	# Add the url as a source: tag
	my $tag = "source:$og_url";
	# Hand off the result to handle_incoming_file
	my ( $status, $id, $title, $message ) = LANraragi::Model::Upload::handle_incoming_file( $tempfile, $catid, $tag );

[xiazeyu]

Found the source of the problem:
source tag is conflict with the "Add Timestamp Tag" option.
When "Add Timestamp Tag" is on, it will directly overwrite the "source" tag instead of append to it.

[xiazeyu]

I'm not sure if it is because of this:

LANraragi/lib/LANraragi/Utils/Database.pm

Lines 60 to 70 in 543609b

	if ( LANraragi::Model::Config->enable_dateadded eq "1" ) {
	$logger->debug("Adding timestamp tag...");
	if ( LANraragi::Model::Config->use_lastmodified eq "1" ) {
	$logger->info("Using file date");
	my $date = ( stat( $redis->hget( $id, "file" ) ) )[9]; #9 is the unix time stamp for date modified.
	$redis->hset( $id, "tags", "date_added:$date" );
	} else {
	$logger->info("Using current date");
	$redis->hset( $id, "tags", "date_added:" . time() );

Using hset causes redis to overwrite existing tags

[xiazeyu]

I found in this commit, DateAdded plugin was fully deprecated. However, considering that DateAdded plugin still exists, and the actual thing it do is a tag plugin, can I revert this commit, and remove "Add Timestamp Tag" from config pannel, and move "Use "Last modified" Time" to plugin's config?

In a word, reuse the DateAdded plugin?

32cc991

[Difegue]

Thanks for the detailed investigation: I don't plan on removing the built-in timestamp feature since third-party clients have come to rely on date_added for extra features, and forcing this back into plugin status would basically just be a regression. 😅

I've pushed out a simple fix so that you can have source: and date_added: tags added simultaneously without issues.