pages:extendToSubpages not recognized

[seirerman]

When I deactivate a page and additionally set the flag extendToSubpages to 1 (quite handy if I want to hide a big part of a website), its subpages will be deactivated, too.

But Solr still lists them in the search results, which leads to 404 errors.
Since the records of the subpages aren't changed, Solr doesn't notice this, and the scheduler task remains at 100%.

Clearing the index and reindexing all pages at least removes the hidden pages from the search results, but also leads to indexing errors for all those subpages:

1319116885: exception 'RuntimeException' with message 'Failed to execute Page Indexer Request. See log for details. Request ID: 55c1b8c640395' in /var/www/html/t3kons/typo3conf/ext/solr/Classes/IndexQueue/PageIndexerRequest.php:163
Stack trace:
#0 /var/www/html/t3kons/typo3conf/ext/solr/Classes/IndexQueue/PageIndexer.php(409): Tx_Solr_IndexQueue_PageIndexerRequest->send('https://www.kon...')
#1 /var/www/html/t3kons/typo3conf/ext/solr/Classes/IndexQueue/PageIndexer.php(55): Tx_Solr_IndexQueue_PageIndexer->getAccessGroupsFromContent(Object(Tx_Solr_IndexQueue_Item), 0)
#2 /var/www/html/t3kons/typo3conf/ext/solr/Scheduler/IndexQueueWorkerTask.php(115): Tx_Solr_IndexQueue_PageIndexer->index(Object(Tx_Solr_IndexQueue_Item))
#3 /var/www/html/t3kons/typo3conf/ext/solr/Scheduler/IndexQueueWorkerTask.php(78): Tx_Solr_Scheduler_IndexQueueWorkerTask->indexItem(Object(Tx_Solr_IndexQueue_Item))
#4 /var/www/html/t3kons/typo3conf/ext/solr/Scheduler/IndexQueueWorkerTask.php(57): Tx_Solr_Scheduler_IndexQueueWorkerTask->indexItems()
#5 /var/www/typo3/typo3-6.2.14/typo3/sysext/scheduler/Classes/Scheduler.php(148): Tx_Solr_Scheduler_IndexQueueWorkerTask->execute()
#6 /var/www/typo3/typo3-6.2.14/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(873): TYPO3\CMS\Scheduler\Scheduler->executeTask(Object(Tx_Solr_Scheduler_IndexQueueWorkerTask))
#7 /var/www/typo3/typo3-6.2.14/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(191): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->executeTasks()
#8 /var/www/typo3/typo3-6.2.14/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(137): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->getModuleContent()
#9 /var/www/typo3/typo3-6.2.14/typo3/sysext/scheduler/mod1/index.php(22): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->main()
#10 /var/www/typo3/typo3-6.2.14/typo3/mod.php(32): require('/var/www/typo3/...')
#11 {main}

[irnnr]

AFAIR extendToSubpages used to not be applied to the hidden field - and I would be surprised if it actually did now; at least I did not see any change like that. It should only apply to the FE user access restrictions. (I just tested it with 6.2 and that behavior is not reproducable).

What TYPO3 version are you on? Can you please make sure this behavior is not caused by another extension you may have installed.
That said, it is a known issue that recursive changes are not being picked up, but that's pretty much a core issue of not propagating a recursive change and not calling hooks for affected sub-pages.

[irnnr]

I was able to reproduce this now when using the hidden field. In my previous comment I was thinking about the "Hide in Menu" field. I'm surprised nobody else ran into this yet. The issue about recursive changes remains though.

[timohund]

I'll now work on this

[timohund]

@irnnr
@seirerman

I've worked on this issue and implemented a functionality in the GarbageCollector and RecordMonitor to apply recursive deletion or reindexing under certain conditions. I've added the above mentioned "triggers".

It would be nice when you can check if this resolved you problem / requirments.

PR can be found here: #192

[seirerman]

I tested your patch, and while I'm thankful for your effort, we're not quite there yet:

My test pages:
Page A

• Subpage B

When I deactivate page A, page B stays in the index. (This is how it should be)
When I deactivate page A and set the extendToSubpages flag to 1, pages A and B are removed from the index. (This is how it should be)
Page B is now back on the index queue, but on the next indexer run l get an exception:

1319116885: exception 'RuntimeException' with message 'Failed to execute Page Indexer Request. See log for details.
Request ID: 56790aa8edb09' in /var/www/html/t3inn/typo3conf/ext/solr/Classes/IndexQueue/PageIndexerRequest.php:180
Stack trace:
#0 /var/www/html/t3inn/typo3conf/ext/solr/Classes/IndexQueue/PageIndexer.php(167): ApacheSolrForTypo3\Solr\IndexQueue\PageIndexerRequest->send('https://www.der...')
#1 /var/www/html/t3inn/typo3conf/ext/solr/Classes/IndexQueue/PageIndexer.php(62): ApacheSolrForTypo3\Solr\IndexQueue\PageIndexer->getAccessGroupsFromContent(Object(ApacheSolrForTypo3\Solr\IndexQueue\Item), 0)
#2 /var/www/html/t3inn/typo3conf/ext/solr/Classes/Task/IndexQueueWorkerTask.php(128): ApacheSolrForTypo3\Solr\IndexQueue\PageIndexer->index(Object(ApacheSolrForTypo3\Solr\IndexQueue\Item))
#3 /var/www/html/t3inn/typo3conf/ext/solr/Classes/Task/IndexQueueWorkerTask.php(90): ApacheSolrForTypo3\Solr\Task\IndexQueueWorkerTask->indexItem(Object(ApacheSolrForTypo3\Solr\IndexQueue\Item))
#4 /var/www/html/t3inn/typo3conf/ext/solr/Classes/Task/IndexQueueWorkerTask.php(69): ApacheSolrForTypo3\Solr\Task\IndexQueueWorkerTask->indexItems()
#5 /var/www/typo3/typo3-6.2.17/typo3/sysext/scheduler/Classes/Scheduler.php(148): ApacheSolrForTypo3\Solr\Task\IndexQueueWorkerTask->execute()
#6 /var/www/typo3/typo3-6.2.17/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(873): TYPO3\CMS\Scheduler\Scheduler->executeTask(Object(ApacheSolrForTypo3\Solr\Task\IndexQueueWorkerTask))
#7 /var/www/typo3/typo3-6.2.17/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(191): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->executeTasks()
#8 /var/www/typo3/typo3-6.2.17/typo3/sysext/scheduler/Classes/Controller/SchedulerModuleController.php(137): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->getModuleContent()
#9 /var/www/typo3/typo3-6.2.17/typo3/sysext/scheduler/mod1/index.php(22): TYPO3\CMS\Scheduler\Controller\SchedulerModuleController->main()
#10 /var/www/typo3/typo3-6.2.17/typo3/mod.php(32): require('/var/www/typo3/...')
#11 {main}

This also means that even if i set extendToSubpages back to 0 on page A, page B can't be indexed anymore. I'd have to manually delete page B from the index queue table in the database for that to happen again.

[timohund]

@seirerman Thanks for your Feedback. Which TYPO3 Version are you using? Can you maybe check what is in the log? Here the same use case is working. I'll try to reproduce this.

[seirerman]

We're on 6.2.17.

Devlog says it got a 404 from page B which is correct.

[timohund]

@seirerman I've adjusted the patch and checked it on my installation. Can you maybe check again, if everything is fine now?

[seirerman]

Yes! Looks good to me.