Mass Tag Edit from CSV (or similar)

[sergalbutt]

When the tag categories extension got introduced, I found myself having to re-tag a buttload of images via the 'Mass tag edit' admin tool because existing images don't use the aliases I set (or at least they didn't when I tried)

Having very little knowledge of PHP, I just can't code something myself, nor can I work out how the hell to do it via MySQL.

Having something similar to the tag alias CSV upload thing but to re-tag images would be fantastic, but I can settle with someone just telling me how to do it on a mass scale via a PHP form or SQL query would be just as helpful.

[muhkuh2005]

For me it always worked instantly when i setup a new alias oO
Did you load the alias list via CSV and they didnt update then?

[sergalbutt]

Yeah.

[sergalbutt]

But I really only noticed that it didn't change on the more common tags, like the ones with counts over 100+

[muhkuh2005]

ugh, my shimmie (closed and only people who i invite can join; might make that an extension someday~) doesnt have any tag which occurs more than 80 times so i can't test that myself

[sergalbutt]

It's not an exact number, it's just a rough estimate. It could have been less, could have been more.

[espressoelf]

Theoretically speaking, adding something like

$aae = new AddAliasEvent($pair['oldtag'], $pair['newtag']);
send_event($aae);

after line 144 of ext/alias_editor/main.php should retag on every new pair. And for the already added aliases you need to add a button or something like that going over all images =/

[sergalbutt]

Could the fact that the CSV I uploaded contained just over 300 new aliases be an issue as to why things didn't re-tag?

[espressoelf]

Well I did a mistake above, don't use the code. And nope, as far as I can tell when adding a single alias, it updates the tags and saves the pair into the DB. When uploadig a CSV, it just saves the pairs in the DB and doesn't change anything.

[sergalbutt]

Ah I see. So this is (technically) a bug then?

[muhkuh2005]

Kinda, but i guess its more about the max_execution time of php which could turn out to be a problem if its set too strict and there are too many images already

[sergalbutt]

@HungryFeline I'm not sure if you mis-phrased or if I misread but did you mean that when a CSV alias list gets uploaded, it doesn't do anything to existing tags like as if I were to add the alias manually?

And to either of you, could you suggest a workaround?

[espressoelf]

When adding with a CSV only new images will get the tag replaced. I made something working but don't know if it's a dirty hack or fit for the main repo so I'm just pasting it here:

File ext/alias_editor/main.php replace

                $pair = array("oldtag" => $parts[0], "newtag" => $parts[1]);
                if(!$database->get_row("SELECT * FROM aliases WHERE oldtag=:oldtag AND lower(newtag)=lower(:newtag)", $pair)){
                    if(!$database->get_row("SELECT * FROM aliases WHERE oldtag=:newtag", array("newtag" => $pair['newtag']))){
                        $database->execute("INSERT INTO aliases(oldtag, newtag) VALUES(:oldtag, :newtag)", $pair);
                    }
                }

with

                $aae = new AddAliasEvent($parts[0], $parts[1]);
                send_event($aae);

I tagged 2 different images, one with "test1", one with "test2" and added a CSV with the value

test1,test0
test2,test0

and it worked. That's all testing done. Don't expect too much from the code :D

[espressoelf]

Also note that this could've some unexpected side-effects when an error occurs. At least the import would be canceled.

[sergalbutt]

Well I always back up my database before attempting any of this kind of stuff anyway :P
I'll let you know how it goes!

[muhkuh2005]

with how fast things are going right now it kinda feels like we're in some kinda chatroom :> iLike

[sergalbutt]

It's because we don't have better things to be doing 😆

[muhkuh2005]

You dont? Well I'm moderating my online radio show right now thehe, but i kinda like how people get together here, i prefer smaller communites over those big ones where noone know the other and people just rant around~

[espressoelf]

@sergalbutt I'd love to hear how it worked after you tried. I hope it works like intended, even if it may be a dirty hack :D

@muhkuh2005 As long as we actively discuss the issue, I don't see anything wrong with that. If we start to talk about our anime collection however, it'd be offtopic and most likely unwanted.

I'm out of this one for now, tell me if it worked :) I also added a public email address if you'd like to contact me.

[sergalbutt]

@HungryFeline It didn't work 😢

[sergalbutt]

@muhkuh2005 The better thing I could be doing right now is sleeping. It's nearly 1am here. But instead I come here with a n00bish problem for this wonderful piece of software.

[espressoelf]

What was the problem? Did it show any error message?

[sergalbutt]

@HungryFeline Nope, but I don't have the PHP error display thing on. Stupid me.

Also, I gave it a try with 2 tags, the first a count 1002 and the second a count of 51

[sergalbutt]

And I don't actually remember how to get PHP errors to display in-browser...

[sergalbutt]

Wait I found it. Let me give it another go...

My apologies for being a 'tard about this...

[sergalbutt]

@HungryFeline Boom.

Fatal error: Maximum execution time of 300 seconds exceeded in S:\XAMPP\htdocs\shimmie\core\database.class.php on line 391

[muhkuh2005]

Thats what i was afraid off, the max execution time...
You can either set the max execution time higher ( http://www.php.net/manual/en/info.configuration.php#ini.max-execution-time ) or simply do it manually~

[sergalbutt]

I have no idea what I would set it to. It could take days with all the tags I have D:

F*** it. I'm going to bed and I will continue this after I wake.

Thanks for your help @muhkuh2005 & @HungryFeline

[espressoelf]

Since max_execution_time can be changed anywhere, you could just add

ini_set('max_execution_time',0);

to your data/config/shimmie.conf.php

[muhkuh2005]

@HungryFeline only works with safe_mode beeing deactivated

[espressoelf]

http://www.php.net/manual/en/features.safe-mode.php

This feature has been DEPRECATED as of PHP 5.3.0 and REMOVED as of PHP 5.4.0.

I'm not on a shared hoster so I've never used it anyways. Setting open_basedir and disabling certain functions is enough imho.

sergalbutt shouldn't have problems, too. XAMPP has this disabled (and should also have a newer PHP version).

[sergalbutt]

I'm not sure what version of PHP I'm running, but safe_mode exists and it is deactivated because it's the only way I could get nice URLs to work.

[sergalbutt]

SO I test it in 4 different ways, and this handy little table shows how many tags were used and the total tag count.

    +------+-------+
    |      | Total |
    | Tags | Count |
+---+------+-------+
| 1 |    5 |    25 |
+---+------+-------+
| 2 |    6 |   123 |
+---+------+-------+
| 3 |    7 |   585 |
+---+------+-------+
| 4 |    6 |   835 |
+---+------+-------+

tl;dr It works. The more tags you feed it, the slower it is (but duh, right? 👅) but it works.

Thanks again for all your help~ 😄

[espressoelf]

Glad to hear it's working. Well, you're dealing with a lot of data, normally, this should be done with a CLI script and not over the browser.

Like packing the images into a zip file. Even my Shimmie has 55,353 images (~50G)...