New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~100s of unfinished Cron Jobs after every post update? #2843

Closed
Crunchify opened this Issue Aug 14, 2015 · 34 comments

Comments

Projects
None yet
@Crunchify

Crunchify commented Aug 14, 2015

Hi Yoast team -

Noticed ~100s of unfinished cron jobs which I believe got created after every post update.. I suppose to check my cron jobs dashboard and surprisingly noticed the same..

Looks like a bug. Let me know if you need more info.

@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Aug 14, 2015

Contributor

Is it after saving post once or repeatedly?

It's normal for plugin to schedule these on saves, though I do agree it looks like some de-duplication is in order.

Contributor

Rarst commented Aug 14, 2015

Is it after saving post once or repeatedly?

It's normal for plugin to schedule these on saves, though I do agree it looks like some de-duplication is in order.

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Aug 14, 2015

this happens after every post update/publish action. For example: if you update (small changes to post and click update button) a post then it will create 3 scheduled tasks..

do_pings
wpseo_hit_sitemap_index
wpseo_ping_search_engines

Which is valid action but somehow it's not getting completed. Instead it should finish within seconds.

likewise - if you update 2 posts then you will see 6 (3x2) unfinished schedule tasks.

Crunchify commented Aug 14, 2015

this happens after every post update/publish action. For example: if you update (small changes to post and click update button) a post then it will create 3 scheduled tasks..

do_pings
wpseo_hit_sitemap_index
wpseo_ping_search_engines

Which is valid action but somehow it's not getting completed. Instead it should finish within seconds.

likewise - if you update 2 posts then you will see 6 (3x2) unfinished schedule tasks.

@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Aug 14, 2015

Contributor

It's not finishing within seconds because it's getting scheduled 5 minutes ahead, as of current state of code.

Contributor

Rarst commented Aug 14, 2015

It's not finishing within seconds because it's getting scheduled 5 minutes ahead, as of current state of code.

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Aug 14, 2015

Ack @Rarst. Just to avoid confusion - it's not getting finished after 5 min also :)

Crunchify commented Aug 14, 2015

Ack @Rarst. Just to avoid confusion - it's not getting finished after 5 min also :)

@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Aug 14, 2015

Contributor

Well, not finishing on time would be issue with your cron/installation. :) After we schedule event, it's up to WP to deal with it.

Contributor

Rarst commented Aug 14, 2015

Well, not finishing on time would be issue with your cron/installation. :) After we schedule event, it's up to WP to deal with it.

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Aug 30, 2015

Closing this issue. Turn out to be an issue with setting in .htaccess file.

Crunchify commented Aug 30, 2015

Closing this issue. Turn out to be an issue with setting in .htaccess file.

@Crunchify Crunchify closed this Aug 30, 2015

@Rarst Rarst reopened this Aug 30, 2015

@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Aug 30, 2015

Contributor

I want to keep this and look to de-duplicate those still. :) Sitemap stuff is on my plate now for big refactor/cleanup #2787

Contributor

Rarst commented Aug 30, 2015

I want to keep this and look to de-duplicate those still. :) Sitemap stuff is on my plate now for big refactor/cleanup #2787

@VeganOstomy

This comment has been minimized.

Show comment
Hide comment
@VeganOstomy

VeganOstomy Oct 1, 2015

I'm having the same problem - @SmashrOrg What setting needed to be changed in your .htaccess file??

wpseo cron jobs

VeganOstomy commented Oct 1, 2015

I'm having the same problem - @SmashrOrg What setting needed to be changed in your .htaccess file??

wpseo cron jobs

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Oct 1, 2015

Hi @VeganOstomy. I removed below custom setting from my .htaccess file and issue resolved.

RewriteEngine On RewriteCond %{QUERY_STRING} !="" RewriteCond %{QUERY_STRING} !^s=.* RewriteCond %{QUERY_STRING} !^q=.* RewriteCond %{REQUEST_URI} !^/wp-admin.* RewriteRule ^(.*)$ /$1? [R=301,L]

Crunchify commented Oct 1, 2015

Hi @VeganOstomy. I removed below custom setting from my .htaccess file and issue resolved.

RewriteEngine On RewriteCond %{QUERY_STRING} !="" RewriteCond %{QUERY_STRING} !^s=.* RewriteCond %{QUERY_STRING} !^q=.* RewriteCond %{REQUEST_URI} !^/wp-admin.* RewriteRule ^(.*)$ /$1? [R=301,L]
@VeganOstomy

This comment has been minimized.

Show comment
Hide comment
@VeganOstomy

VeganOstomy Oct 1, 2015

@SmashrOrg Interesting. Mine looks way different:

RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

I cleared all those old cron jobs yesterday, and haven't seen any new ones, but they will come back - they always seem to, and I don't know what triggers them.

VeganOstomy commented Oct 1, 2015

@SmashrOrg Interesting. Mine looks way different:

RewriteEngine On
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

I cleared all those old cron jobs yesterday, and haven't seen any new ones, but they will come back - they always seem to, and I don't know what triggers them.

@Ciantic

This comment has been minimized.

Show comment
Hide comment
@Ciantic

Ciantic Dec 15, 2015

I'm seeing a lot of these:

  [1449148047]=>
  array(1) {
    ["wpseo_ping_search_engines"]=>
    array(1) {
      ["40cd750bba9870f18aada2478b24840a"]=>
      array(2) {
        ["schedule"]=>
        bool(false)
        ["args"]=>
        array(0) {
        }
      }
    }
  }

with var_dump(_get_cron_array()); where date (the integer key) is in way past, two days backwards. What is going on here?

They should've been run already and being removed from cron, dates are: 12/11/2015 @ 1:02pm (UTC) etc...

Ciantic commented Dec 15, 2015

I'm seeing a lot of these:

  [1449148047]=>
  array(1) {
    ["wpseo_ping_search_engines"]=>
    array(1) {
      ["40cd750bba9870f18aada2478b24840a"]=>
      array(2) {
        ["schedule"]=>
        bool(false)
        ["args"]=>
        array(0) {
        }
      }
    }
  }

with var_dump(_get_cron_array()); where date (the integer key) is in way past, two days backwards. What is going on here?

They should've been run already and being removed from cron, dates are: 12/11/2015 @ 1:02pm (UTC) etc...

@Ciantic

This comment has been minimized.

Show comment
Hide comment
@Ciantic

Ciantic Dec 15, 2015

I think yoast should not just schedule blindly these events, when one is in queue:

$hook = "wpseo_ping_search_engines";
if (!wp_next_scheduled($hook)) {
    wp_schedule_single_event(time() + 3600 * 1, $hook);
}

Above is example code, which should be done in the plugin, so it won't just add more and more of those hooks when old ones are not even run.

I consider this as a bug, why would you schedule wpseo_ping_search_engines several times even if old ones are not run?

Ciantic commented Dec 15, 2015

I think yoast should not just schedule blindly these events, when one is in queue:

$hook = "wpseo_ping_search_engines";
if (!wp_next_scheduled($hook)) {
    wp_schedule_single_event(time() + 3600 * 1, $hook);
}

Above is example code, which should be done in the plugin, so it won't just add more and more of those hooks when old ones are not even run.

I consider this as a bug, why would you schedule wpseo_ping_search_engines several times even if old ones are not run?

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify commented Dec 15, 2015

+1. Agree with @Ciantic

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost commented Jan 20, 2016

@chuckreynolds

This comment has been minimized.

Show comment
Hide comment
@chuckreynolds

chuckreynolds Apr 11, 2016

Contributor

status of this? just had this problem. big problem.

Contributor

chuckreynolds commented Apr 11, 2016

status of this? just had this problem. big problem.

@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Apr 12, 2016

Contributor

The big refactoring of sitemaps landed for #2787 and now actively working through any and all open sitemap issues. :) So getting to this soon.

Contributor

Rarst commented Apr 12, 2016

The big refactoring of sitemaps landed for #2787 and now actively working through any and all open sitemap issues. :) So getting to this soon.

@chuckreynolds

This comment has been minimized.

Show comment
Hide comment
@chuckreynolds

chuckreynolds Apr 12, 2016

Contributor

What are the chances this happened because Google Search Console was no longer authorized?
This seems to only be happening on a staging site that the DB was migrated over from production (the url verified with GSC). testing now since I toggled it off on the staging site by clicking reauth. (Also no way to unhook that otherwise?).

Also adding this to WP_ENV == staging to see if that kills it all;

add_filter( 'wpseo_allow_xml_sitemap_ping', '__return_false' );
Contributor

chuckreynolds commented Apr 12, 2016

What are the chances this happened because Google Search Console was no longer authorized?
This seems to only be happening on a staging site that the DB was migrated over from production (the url verified with GSC). testing now since I toggled it off on the staging site by clicking reauth. (Also no way to unhook that otherwise?).

Also adding this to WP_ENV == staging to see if that kills it all;

add_filter( 'wpseo_allow_xml_sitemap_ping', '__return_false' );
@Rarst

This comment has been minimized.

Show comment
Hide comment
@Rarst

Rarst Apr 12, 2016

Contributor

I think pings are anonymous, they shouldn't care about authorization. They were implemented long before GSC.

Contributor

Rarst commented Apr 12, 2016

I think pings are anonymous, they shouldn't care about authorization. They were implemented long before GSC.

@chuckreynolds

This comment has been minimized.

Show comment
Hide comment
@chuckreynolds

chuckreynolds Apr 12, 2016

Contributor

agreed... i put the filter in if environment != production so technically... the cron should stop trying to ping... i'll update in a bit. using wp-crontrol plugin to monitor.

edit: seems to be okay w/ that filter in place. no idea why that sitemap ping is causing so many problems.

Contributor

chuckreynolds commented Apr 12, 2016

agreed... i put the filter in if environment != production so technically... the cron should stop trying to ping... i'll update in a bit. using wp-crontrol plugin to monitor.

edit: seems to be okay w/ that filter in place. no idea why that sitemap ping is causing so many problems.

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify commented May 10, 2016

Thanks.

@brianhogg

This comment has been minimized.

Show comment
Hide comment
@brianhogg

brianhogg Jan 3, 2017

This seems to still be an issue on the latest version 4.0.2 where hundreds are created but never complete. I'm running Easy Digital Downloads so perhaps they are being created on the back of sales entries or something? With no arguments I'm not seeing what is triggering their creation.

Is the solution to add the filter on wpseo_allow_xml_sitemap_ping to every install with Yoast SEO or should it no longer be happening?

screenshot 2017-01-02 20 23 55

brianhogg commented Jan 3, 2017

This seems to still be an issue on the latest version 4.0.2 where hundreds are created but never complete. I'm running Easy Digital Downloads so perhaps they are being created on the back of sales entries or something? With no arguments I'm not seeing what is triggering their creation.

Is the solution to add the filter on wpseo_allow_xml_sitemap_ping to every install with Yoast SEO or should it no longer be happening?

screenshot 2017-01-02 20 23 55

@brianhogg

This comment has been minimized.

Show comment
Hide comment
@brianhogg

brianhogg Jan 3, 2017

Apologies - looks like this may have been due to WP Engine's caching system and the previous cron entries added by an older version of SEO. Will update if they reappear under 4.0.2.

brianhogg commented Jan 3, 2017

Apologies - looks like this may have been due to WP Engine's caching system and the previous cron entries added by an older version of SEO. Will update if they reappear under 4.0.2.

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Jan 3, 2017

@brianhogg thanks for looking at this. I could still confirm an issue.

Solution might be very simple.
--> If cron jobs Key exist in queue then just don't create new one :) What do you think?

Crunchify commented Jan 3, 2017

@brianhogg thanks for looking at this. I could still confirm an issue.

Solution might be very simple.
--> If cron jobs Key exist in queue then just don't create new one :) What do you think?

@brianhogg

This comment has been minimized.

Show comment
Hide comment
@brianhogg

brianhogg Jan 3, 2017

That would be a good solution yes :) I've since cleared out all the pending cron jobs, and there's no 'stuck' yoast jobs present anymore. So might be that for those who had this issue before, some way to detect it and clear out the scheduled tasks and it's no longer an issue in the current version?

There's also an issue it seems with WP Engine and cron jobs when their object caching layer is enabled. This might be preventing both the scheduled tasks from clearing and preventing Yoast from detecting they already exist. Other hosts with various caching layers or those using a caching plugin might see the same thing.

brianhogg commented Jan 3, 2017

That would be a good solution yes :) I've since cleared out all the pending cron jobs, and there's no 'stuck' yoast jobs present anymore. So might be that for those who had this issue before, some way to detect it and clear out the scheduled tasks and it's no longer an issue in the current version?

There's also an issue it seems with WP Engine and cron jobs when their object caching layer is enabled. This might be preventing both the scheduled tasks from clearing and preventing Yoast from detecting they already exist. Other hosts with various caching layers or those using a caching plugin might see the same thing.

@Ciantic

This comment has been minimized.

Show comment
Hide comment
@Ciantic

Ciantic Jan 3, 2017

Solution is still the same thing as it was in 2015,

Use this pattern, it's used extensively in other cron things too:

$hook = "wpseo_ping_search_engines";
if (!wp_next_scheduled($hook)) {
    wp_schedule_single_event(time() + 3600 * 1, $hook);
}

Ciantic commented Jan 3, 2017

Solution is still the same thing as it was in 2015,

Use this pattern, it's used extensively in other cron things too:

$hook = "wpseo_ping_search_engines";
if (!wp_next_scheduled($hook)) {
    wp_schedule_single_event(time() + 3600 * 1, $hook);
}
@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify Jan 6, 2017

@Ciantic +1 for it.

@brianhogg - it's about additional check if that cron job already exist in queue. Again, in my case I've disabled all wordpress cron jobs and that's why I'm seeing all duplicate cron jobs in queue :)

Crunchify commented Jan 6, 2017

@Ciantic +1 for it.

@brianhogg - it's about additional check if that cron job already exist in queue. Again, in my case I've disabled all wordpress cron jobs and that's why I'm seeing all duplicate cron jobs in queue :)

@Crunchify

This comment has been minimized.

Show comment
Hide comment
@Crunchify

Crunchify commented Jan 6, 2017

@brianhogg

This comment has been minimized.

Show comment
Hide comment
@brianhogg

brianhogg Jan 6, 2017

@Crunchify the issue came up due to a WP Engine issue with cron jobs, which a lot of people seem to be having, and will hopefully be sorted soon. Definitely seems worth checking if the cron job already exists as per @Ciantic's suggestion regardless :)

brianhogg commented Jan 6, 2017

@Crunchify the issue came up due to a WP Engine issue with cron jobs, which a lot of people seem to be having, and will hopefully be sorted soon. Definitely seems worth checking if the cron job already exists as per @Ciantic's suggestion regardless :)

@KwadroNaut

This comment has been minimized.

Show comment
Hide comment
@KwadroNaut

KwadroNaut Jan 22, 2017

@brianhogg can you please include a pointer to 'the WP Engine issue'? Thanks.

KwadroNaut commented Jan 22, 2017

@brianhogg can you please include a pointer to 'the WP Engine issue'? Thanks.

@brianhogg

This comment has been minimized.

Show comment
Hide comment
@brianhogg

brianhogg Jan 23, 2017

@KwadroNaut they only created an internal ticket for it, sorry. Essentially the WP cron was not firing when it should for days at a time. Appears to be sorted out now though.

brianhogg commented Jan 23, 2017

@KwadroNaut they only created an internal ticket for it, sorry. Essentially the WP cron was not firing when it should for days at a time. Appears to be sorted out now though.

@MrAlexWeber

This comment has been minimized.

Show comment
Hide comment
@MrAlexWeber

MrAlexWeber Jun 8, 2017

Hi, just ran into this issue as well—many many wp-cron entries for Yoast SEO. I'm not sure how to remove them! Plugin is on version 2.3.5

MrAlexWeber commented Jun 8, 2017

Hi, just ran into this issue as well—many many wp-cron entries for Yoast SEO. I'm not sure how to remove them! Plugin is on version 2.3.5

@KwadroNaut

This comment has been minimized.

Show comment
Hide comment
@KwadroNaut

KwadroNaut Jun 9, 2017

@MrAlexWeber wp_cli is your friend:
wp_clear_scheduled_hook( 'wpseo_ping_search_engines' );
wp_clear_scheduled_hook( 'do_pings' );

cron event list will tell you how many things are due and when, the manual should be good enough to figure out the rest.

KwadroNaut commented Jun 9, 2017

@MrAlexWeber wp_cli is your friend:
wp_clear_scheduled_hook( 'wpseo_ping_search_engines' );
wp_clear_scheduled_hook( 'do_pings' );

cron event list will tell you how many things are due and when, the manual should be good enough to figure out the rest.

@chuckreynolds

This comment has been minimized.

Show comment
Hide comment
@chuckreynolds
Contributor

chuckreynolds commented Jun 9, 2017

@andizer andizer removed their assignment Jul 4, 2017

@aosmichenko

This comment has been minimized.

Show comment
Hide comment
@aosmichenko

aosmichenko Mar 23, 2018

Now the other wpseo_hit_sitemap_index is spammed and killing our database..

aosmichenko commented Mar 23, 2018

Now the other wpseo_hit_sitemap_index is spammed and killing our database..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment