staticpublisher queued job fails in some circumstances

[lars-lemon8]

Running into problems with SS4.11 / Fluent V6 in combination with the Static Publisher.

Situation:

• Fluent 6
• SS 4.11
• Running in CLI
• Empty Job queue
• No configuration added for staticpublisher (no urlsToCache() ).

Run StaticCacheFullBuildTask --> added to the queue for immediate processing
Run ProcessJobQueueTask --> Argument #1 ($value) must be of type Countable|array, null given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, null given at /data/site/vendor/silverstripe/staticpublishqueue/src/Job.php:102)"

When I re-publish a page in the CMS, it gets added to the queue and publishes correctly if I process the queue again.

If I try to run publisher tasks in the CMS jobs UI itself , the task runs, but takes a loooooong time and kills the session. Will have to look at the logs to see what is happening.

[guci0]

Hi Lars,
first of all check your Job file and compare to:

silverstripe-staticpublishqueue/src/Job/StaticCacheFullBuildJob.php

Line 43 in 2f05ec1

public function setup(): void

There is a difference than via composer install, I don't know why...

NEXT We need a little fix:

<?php
namespace Mynamespace;

use SilverStripe\CMS\Model\SiteTree;
use SilverStripe\StaticPublishQueue\Contract\StaticallyPublishable;
use SilverStripe\StaticPublishQueue\Extension\Publishable\PublishableSiteTree;
use SilverStripe\StaticPublishQueue\Job\StaticCacheFullBuildJob;
use SilverStripe\Versioned\Versioned;
use TractorCow\Fluent\State\FluentState;

class StaticPublisher extends StaticCacheFullBuildJob {

    protected function getAllLivePageURLs(): array {
        $urls = [];
        $this->extend('beforeGetAllLivePageURLs', $urls);

        foreach (Locale::getLocales() as $locale) {

            $state = FluentState::singleton();
            $state->setLocale($locale->getLocale());
            $livePages = Versioned::get_by_stage(SiteTree::class, Versioned::LIVE);

            foreach ($livePages as $page) {
                if ($page->hasExtension(PublishableSiteTree::class) || $page instanceof StaticallyPublishable) {
                    $urls = array_merge($urls, $page->urlsToCache());
                }
            }

        }

        $this->extend('afterGetAllLivePageURLs', $urls);
        // @TODO look here when integrating subsites
        // if (class_exists(Subsite::class)) {
        //     Subsite::disable_subsite_filter(true);
        // }
        return $urls;
    }
}

---
Name: staticpublisher for fluent
---
SilverStripe\Core\Injector\Injector:
  Silverstripe\StaticPublishQueue\StaticCacheFullBuildJob:
    class: Mynamespace\StaticPublisher

After that we need some fix to .htaccess in /public

	# Config for site in a subdirectory
	# (remove the following four rules if site is on root of a domain. E.g. test.com rather than test.com/my-site)
	# Cached content - sub-pages (site in sub-directory)
	RewriteCond %{REQUEST_METHOD} ^GET|HEAD$
	RewriteCond %{QUERY_STRING} ^$
	#RewriteCond %{REQUEST_URI} /(.*[^/])/(.*[^/])/?$
	RewriteCond %{REQUEST_URI} /(.*[^/])/(.*[^/])/?/(.*[^/])/?$
	RewriteCond %{DOCUMENT_ROOT}/%1/cache/%2/%3.html -f
	RewriteCond %{REQUEST_FILENAME} !-f
	RewriteRule .* cache/%2/%3.html [L]

	# Cached content - homepage (site in sub-directory)
	RewriteCond %{REQUEST_METHOD} ^GET|HEAD$
	RewriteCond %{QUERY_STRING} ^$
	RewriteCond %{REQUEST_URI} ^/(.*[^/])/?$
	RewriteCond %{DOCUMENT_ROOT}/%1/cache/index.html -f
	RewriteCond %{REQUEST_FILENAME} !-f
	RewriteRule .* cache/index.html [L]

I don't know it's a best solution but works – I guess, I didn't tests this so much yet.

PS. In my opinion, it should work OUT OF THE BOX, but doesn't.

[GuySartorelli]

@guci0 as I mentioned in your issue I have not been able to reproduce this issue out of the box, and need more information as requested in that issue.

@lars-lemon8 I haven't been able to reproduce your scenario either. I do note that you say "Running in CLI" but then further down you say "If I try to run publisher tasks in the CMS jobs UI itself" so it's not clear whether you're experiencing failures when running the task or processing the queue, and whether you're doing this in the UI or via the task. I have tried all of these scenarios though from a fresh install and can't reproduce your issue. Can you please provide a clear set of step-by-step instructions for how to reproduce this from a fresh install? Please include any composer commands and custom code necessary to reproduce this.

[lars-lemon8]

@GuySartorelli , thanks, sure , will do !

Yeah, everything is in CLI , my remark about the CMS UI was meant as a sidenote , and is confusing, sorry about that.

[lars-lemon8]

@GuySartorelli
I left Fluent out of the equation in this example. My environment is PHP8.

1> Setup SS

• composer create-project silverstripe/installer .
• create DB / setup ENV file
• setup default user / pwd
• dev build -> load site -> works

2> Install static publisher

• composer require silverstripe/staticpublishqueue
• run a dev build

3> Tests (all done from browser, no CLI)

• run /dev/tasks/SilverStripe-StaticPublishQueue-Task-StaticCacheFullBuildTask (in browser), the job gets added with 0/0 tasks

• Press 'play' on job , results in 'broken'

• Messages are empty (which makes sense)
  

• Go to a page in the CMS and publish it (in my case, 'about us'), a job gets added to the queue
  

• Press play on that job, results in no errors , but I get kicked out of the CMS and need to login again.

• However, the publish job has been finished correctly.
  

• run /dev/tasks/queue/SilverStripe-StaticPublishQueue-Task-StaticCacheFullBuildTask, a job gets added to the queue. Run this one. It generates a new full static build job to the queue, however , this one will result in a broken status as well , once played.
  

4> CLI tests

• Run /dev/tasks/SilverStripe-StaticPublishQueue-Task-StaticCacheFullBuildTask results in a
  SilverStripe\StaticPublishQueue\Job\StaticCacheFullBuildJob added to the queue for immediate processing

• Run /dev/tasks/ProcessJobQueueTask results in a
  [2022-10-04 10:07:40] Running Generate static pages for all URLs and others from queue 2. [2022-10-04 10:07:40] error-log.INFO: count(): Argument #1 ($value) must be of type Countable|array, null given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, null given at /data/site/vendor/silverstripe/staticpublishqueue/src/Job.php:102)"} []

I think this is the same issue @guci0 was trying to explain (not being able to do a full build).

Please let me know what other test scenario's I can run through or information I can provide to track this thing down. ;-)

[satrun77]

I have the same error count(): Argument #1 ($value) must be of type Countable|array. Pull request to resolve it #149

[guci0]

I'm already testing on a live server, now there was a problem of access rights (chmods) - files are locked.

/cache and subfolders are fine 775, but .htmls – 600.

Hmmm...

[lars-lemon8]

@guci0: It feels quite unlikely your file permission problems have anything to do with this module.

I have also been testing #149 , it fixes this error for me:

[2022-10-04 10:07:40] Running Generate static pages for all URLs and others from queue 2. [2022-10-04 10:07:40] error-log.INFO: count(): Argument #1 ($value) must be of type Countable|array, null given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, null given at /data/site/vendor/silverstripe/staticpublishqueue/src/Job.php:102)"} []

A new problem arises however; the StaticCacheFullBuildJob now gets stuck in 'waiting' after this message:
error-log.WARNING: Job releasing memory and waiting (256.01 MB used) [] []

However, the funny thing is that it did publish all pages correctly:

Increasing the memory limit does not seem to matter (same error, but higher mem usage), it does the job correctly, but never finishes successfully.

[lars-lemon8]

Okay. New version is released. The results are entirely different now. It got a lot better, but still refuses to complete a StaticCacheFullBuildJob.

Closing this ticket for now. Will spin up a new one with a clean test scenario if I have the time.