Tag index corrupted, need to update at least each day #59

Closed
MajorGrubert opened this Issue Mar 10, 2012 · 18 comments

Comments

Projects
None yet
5 participants

Hi,

I install tagplugin in this wiki : http://www.hybryde.org/wiki/doku.php/

Every day, sometime more, I have to rebuild tagindex beacause it is corrupted.

Often a page inherit tags in topic mode there doesn't have in reality.
Take a look here, note the real tag of this page and the tags mentionned in the topic under :
http://pix.toile-libre.org/upload/original/1331391724.png

We use DokuWiki on our intranet. When I wanted to upgrade to "Angua" and the latest Tag plugin, I stumbled on the same problem. Even a "rebuild tagindex" resulted in a corrupt topic list.

Looking further, it seems the .meta files are corrupted. The 'subject' entry in the meta data often contains multiple entries of the same (correct) tag and occasionaly also wrong tags, resulting in the foremention corruption. I have checked the old meta-data, which seems to be in order, thus concluding that the corruption in caused by the new Tag plug-in in conjunction with "Angua".

I loaded the latest version from git and this solved the problem!

I will try this, thx

I update tag-plugin through admin interface and it seems to solve the pb !
Nice.
I wait a week before close this issue.

This update doesn't solve the problem. I will try the git version.

Owner

michitux commented May 8, 2012

Unless you have a really old installation of the tag plugin updating the plugin should already use the git version. However there is a new version of the tag plugin which uses the metadata index of DokuWiki for storing the tags and no longer uses the old tag plugin database file. It is in the metaindex branch in the git repository. There are still a few problems like missing tag normalization (i.e. tag and Tag are still considered as two different tags) and unfortunately I haven't found the time yet to properly fix that but you can try if it works better for you. You also need the development branch of the cloud plugin and if you use the cumulus plugin you need to use the version on my GitHub account. I hope to fix the remaining problems soon but I don't know if that means within a few days, weeks or maybe also months.

OK, it works with git version. I close this issue ?

Yes, please close it.

In the weeks since installing the GitHub version I have had no more problems, whereas before the metadata was completely corrupted within a space of a few hours.

Greeting,
Hans

-----Oorspronkelijk bericht-----
Van: MajorGrubert [mailto:reply@reply.github.com]
Verzonden: dinsdag 8 mei 2012 22:45
Aan: Hans Sampiemon
Onderwerp: Re: [plugin-tag] Tag index corrupted, need to update at least each day (#59)

OK, it works with git version. I close this issue ?


Reply to this email directly or view it on GitHub:
#59 (comment)

Hi Michael,

When I installed "Angua", I started off with downloading anew all the plug-ins we use via the plug-in page on dokuwiki.org.

The tag plug-in download-button points to the file "dokufreaks-plugin-tag-2010-11-12-30-ga058f92.tar.gz". In hindsight I see this might well be "a really old installation", but it is the most natural place to find the latest downloads. Also the "zip" on the github site points to a file with this same date. It may well be my inexperience with github that I thought this was the latest version.

When I read the pull requests from yoboy and Andreas, I decided to download that version and it solved all my problems.

Greetings,
Hans

-----Oorspronkelijk bericht-----
Van: Michael Hamann [mailto:reply@reply.github.com]
Verzonden: dinsdag 8 mei 2012 21:34
Aan: Hans Sampiemon
Onderwerp: Re: [plugin-tag] Tag index corrupted, need to update at least each day (#59)

Unless you have a really old installation of the tag plugin updating the plugin should already use the git version. However there is a new version of the tag plugin which uses the metadata index of DokuWiki for storing the tags and no longer uses the old tag plugin database file. It is in the metaindex branch in the git repository. There are still a few problems like missing tag normalization (i.e. tag and Tag are still considered as two different tags) and unfortunately I haven't found the time yet to properly fix that but you can try if it works better for you. You also need the development branch of the cloud plugin and if you use the cumulus plugin you need to use the version on my GitHub account. I hope to fix the remaining problems soon but I don't know if that means within a few days, weeks or maybe also months.


Reply to this email directly or view it on GitHub:
#59 (comment)

The problem re-appears slowly.
It seems there still have a corruption in tag-index.
Small : it's the first time I have to rebuild tagindex since my last post (12 days ago).
But the problem still exists.

k4m1 commented Jun 29, 2012

Hello,

Looking further, it seems the .meta files are corrupted. The 'subject' entry in the meta data often contains multiple
entries of the same (correct) tag and occasionaly also wrong tags, resulting in the foremention corruption."

I had the same issue (with 'standard version from dokuwiki tagplugin page'), I have been fighting with this nice couple of days...

Anyway if you change lib/plugins/tag/syntax/tag.php

// merge with previous tags
$renderer->meta['subject'] = array_merge($renderer->meta['subject'], $data);

to just:

$this->tags = $data;

(and then remove metadata, spider wiki and rebuild tagindex) it looks solved. No side effects noticed so far...

Maybe somebody can explain me that?

Best,
Michał

I think it's related to this.

I fixed it locally by adding the following just before line 75 in tag.php:

if (!is_array($renderer->meta['subject'])) $renderer->meta['subject'] = array();
$this->tags = $renderer->meta['subject'];

(I still have the array_merge after this). This was based off the master branch of this repo, which has a different line from what you wrote in tags.php line 75 (maybe you made a typo or changed it yourself before).

The problem is that we can't rely on $this->tags being cleaned up between calls to the renderer, which causes a problem (for example) when creating an index that "meta-renders" a bunch of files all at once. In this case newer versions of Dokuwiki will reuse the same plugin object for a bunch of files (seems like four or five for me) and so the later ones inherit the tags of the earlier ones. My patch just "resets" the $this->tags to the metadata which is guaranteed to be appropriate to the rendered-page (I guess the metaindex branch does this more properly).

If your modification works then I guess it's all good, but I think it fails if one defines tags in multiple places in the same file, because then only the last set will be effective. For the way most people use tags this probably won't even be an issue though.

k4m1 commented Jul 17, 2012

Than you for the explanation.

Yes, I made a typo in my "source line" as I was playing with that to make it work. Should be:

$this->tags = array_merge($this->tags, $data);

to

$this->tags = $data;

I was not even aware that I can define tags in multiple places on one page :) So my fix as a side effect basically removed that functionality ;)

Anyway your fix is better. It is a bit nasty that PHP can treat some 'old junk' as an array, even if that is not an array.

How can we include that to the master branch? Because as a matter of fact without that the plugin is not Compatible with DokuWiki 2012-01-25 "Angua"

Well it's not exactly 'old junk' but stuff that the plugin has put there while rendering a previous page (so it is indeed an array). It is just that when the code was written they assumed there would be one plugin "object" per page meta-render, which must have been true at one point but has now changed. It's more a matter of understanding how Dokuwiki calls its plugins / establishing some best practices (one of which could be "use metadata to store persistent info").

I guess the metaindex branch is addressing this (along with a bunch of more significant changes); if I get around to it maybe I'll do a branch & pull request for this issue but I'm really just at hack-n-slash level when it comes to PHP :)

Owner

michitux commented Jul 17, 2012

The metaindex branch shouldn't have this issue as I've completely removed
the tags array in the syntax plugin. There were some minor issues left that
should be fixed now but I still need to test especially my last changes. If
somebody wants to review and test the changes feel free to do that as I
would like to release the metaindex branch as new version rather sooner
than later but don't have enough time currently.

Owner

michitux commented Jul 25, 2012

I've just done the changes I wanted to do in the metaindex branch and I hope it works now though there might be some bugs left. Could you please try using the metaindex branch? It uses DokuWiki's metadata index instead of the tag plugin's own index, you will need to re-index all pages which will happen automatically when you visit them, you can also rebuild the search index manually either using bin/indexer.php or using the searchindex plugin.

As soon as I'll have got some more feedback that the metaindex branch works I'll merge the changes into the master branch and release it as new version of the tag plugin.

Sorry guys, I can't help you anymore : I've uninstall this plugin, reorganise the wiki and open another site (a dotclear site) beside the older with a efficient tag tool.

Owner

michitux commented Sep 3, 2012

As I assume this is a duplicate of #62 and should be fixed in the new version I'm closing this issue now. If there should still be any problems that haven't been reported in other issues please open a new issue.

michitux closed this Sep 3, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment