Removing a CPT from the sitemap but leaving the CPT Archive page in the sitemap does not work

[rmarcano]

• I've read and understood the contribution guidelines.
• I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened.

Setting a custom post type not to be indexed will remove it from the sitemap and add a "noindex" to all posts in the CPT.

The Yoast plugin has the option of treating the CPT Archive page separately, by letting you index the archive pages. However, selecting "Yes" in "Show CPT Archive in search results" will not add the CPT Archive to the sitemap, even if the pages do not get a "noindex".

Please describe what you expected to happen and why.

I expected that the CPT Archive page would appear in the sitemap.

How can we reproduce this behavior?

1.Create a Custom Post Type.
2.Go to SEO -> Search Appearance -> Content Types
3.Set the Custom Post Type not to appear in search engines.
4.Set the CPT Archive pages to appear in search engines.
5.Verify that the CPT Archive pages are not in the sitemap.

Technical info

• WordPress version: 4.9.8

• Yoast SEO version: 9.0.1

• Tested with theme: TwentySeventeen

[tacoverdo]

Shouldn't we just hide/remove the settings for the CPTs archive when the CPT is set to not show in search results, @jdevalk?

[stodorovic]

@tacoverdo I'm not sure that's the best approach. It's related to #11225. It's possible that someone want to set all posts to "noindex" by global settings (if there are a lot of posts) and manually set "index" to few posts including archive page. In this case there is inconsistency between sitemap and "indexable" posts (search engines should have not problems finding and indexing them, but they may show warnings).

I'll create PR which will fix it in next weeks. Similar issues (which could be fixed): #11225, #9953, #10312. Maybe it's possible to merge all of them into one issue.

[jdevalk]

No there are valid use cases where you'd want the archive indexed, looking forward to that PR @stodorovic :)

[studioavanti]

Hi, any news about this issue, is it fixed now please?

[stodorovic]

It isn't fixed. I've created couple PRs related to sitemaps in meantime and I didn't find time for this issue. I'll try to create PR soon (it could be my next PR related to sitemaps).

[GermanKiwi]

Just found this issue after experiencing the exact problem myself with a CPT. In my case, I deliberately want to not index the single CPT posts, but I also deliberately do want to index (and include in sitemap) the CPT archive page. Therefore I definitely would not like to see the CPT archive setting hidden, as suggested by tacoverdo above. ;) I'm glad you're working on a fix for this!

[ChrisStanyon]

+1 for this. I need to include a CPT Archive, but exclude all the Single Posts. Had to switch to a different Sitemap generator until this is fixed :(

[studioavanti]

Almost 6 months since this issue has been reported, any news about a fix?

[studioavanti]

Any news please?

[Djennez]

This issue currently has no priority and is not being worked on from within our development team. We do welcome any community patches that would fix this though.

Once this is being picked up, this thread will get notified.

[GermanKiwi]

Hi @Djennez, is it likely that this issue will be picked up at some point in the future? Or are you saying that it's a "won't fix" issue for your team? @stodorovic had mentioned, back in January, that he would try and create a PR for it soon. :) As it's effectively a bug of sorts, and @jdevalk has commented that there are valid use-cases for this scenario, then surely it needs to be fixed/implimented? :)

[simonemanfre]

This bug is incredible, the only way I found to solve the problem is to set each post as noindex from the advanced yoast settings of the single post.
Fortunately I have this problem in a post type with 10 posts, but if I had 1000? I would be forced to change the plugin

[shikkaba]

@Djennez It's been a while. Will this issue be fixed anytime in the future?

[Djennez]

I don't believe this has been planned in any of the future projects yet. @JessieHenkes can this possibly be included in the Indexable sitemap overhaul? Or is it too much out of scope for that?

[shikkaba]

@JessieHenkes Any follow up on this?

[Djennez]

As soon as it's worked on / fixed, you'll see this thread being updated.

[shikkaba]

@Djennez Sorry. I don't mean to bother anyone. I know you have lives and such. :)
I was just not sure how to pursue it as this issue was opened in 2018, and it would be awesome to know if it was at least on the radar in some way.

[acerus]

@studioavanti @shikkaba @GermanKiwi I've recently stumbled into this bug and thanks to some new filters in Yoast SEO was able to come up with a quick solution that helped me. Sharing it with you (just change $post_type value to whatever post type archive you want to add):

https://wordpressify.ru/2020/12/dobavlenie-svoej-ssylki-v-sitemap-stranits-yoast-seo/

[GermanKiwi]

@acerus thanks so much for providing that function! It works well, and in the absence of an official fix from the developers, this is a really good workaround. 👍

I've actually made a couple of small adjustments to the function, and I'll paste my version of it below. In short: I noticed that your version was outputting the date in the page-sitemap.xml file in this format:

2020-12-20 05:16

Whereas the dates for every other page in this file use the following format:

2020-12-20T04:16:44+00:00

So in order to get this format, I've tweaked your function to look like this:

function add_archive_URL() {
	$post_type = 'my_custom_post_type';
	$archive_url = get_post_type_archive_link( $post_type );

	$args = [
		'posts_per_page'         => 1,
		'post_type'              => $post_type,
		'orderby'                => 'modified',
		'order'                  => 'DESC',
		'no_found_rows'          => true,
		'update_post_meta_cache' => false,
		'update_post_term_cache' => false,
		'fields'                 => 'ids',
		'cache_results'          => true
	];

	$latest = new WP_Query( $args );

	while ( $latest->have_posts() ) {
		$latest->the_post();
		$date = get_post_modified_time( 'Y-m-d h:i:s', true );
		$last_mod = YoastSEO()->helpers->date->format( $date );
	}

	$url = "\t<url>\n";
	$url .= "\t\t<loc>$archive_url</loc>\n";
	$url .= "\t\t<lastmod>$last_mod</lastmod>\n";
	$url .= "\t</url>\n";

	return $url;

}
add_filter( 'wpseo_sitemap_page_content', 'add_archive_URL' );

Key things I changed:

• Replaced get_the_modified_time with get_post_modified_time so that it would use GMT for the timezone, like the other dates in the XML file.
• Changed the date format from Y-m-d h:i to Y-m-d h:i:s because we need to include seconds.
• Filtered the date through YoastSEO()->helpers->date->format( $date ) which I copied from \wordpress-seo\inc\sitemaps\class-sitemaps-renderer.php - this puts the date in the same format used in the rest of the XML file.
• I also added line breaks and tabs to the $url output, to match the layout of the rest of the XML file.

Hope that's helpful!

[acerus]

@acerus thanks so much for providing that function! It works well, and in the absence of an official fix from the developers, this is a really good workaround. 👍

I've actually made a couple of small adjustments

Thank you, those are actually very helpful adjustments!

[MaxDarklighter]

@acerus , @GermanKiwi what if I need to add several CPT archive pages to my sitemap.xml - do I need to copy/paste our code several times or may be I can just use commas, separating my post_type titles?

[GermanKiwi]

@MaxDarklighter that's a good question. My suspicion is that you may need to make additional copies of the function - one per CPT, with each function having a unique name of course. However, it's entirely possible I'm wrong, and hopefully someone with deeper knowledge of functions could give a more authoritative answer here.

At the very least, it will certainly work fine to use multiple functions. It won't break anything. I just don't know if it's the most efficient way to do it.

[MaxDarklighter]

@GermanKiwi just 10min ago I tried to do it and I got critical error on my site. But no worries - I deleted all copies of "function add_archive_URL()" and it's ok now :)

[GermanKiwi]

@MaxDarklighter if you're adding more than one copy of the function, it's important that each copy has a unique name, which is mentioned in both the 1st line (beginning with "function") and last line (beginning with "add_filter").

So for example:

function add_archive_URL_CPT1() {
	$post_type = 'my_custom_post_type_1';

	(REST OF THE FUNCTION)
}
add_filter( 'wpseo_sitemap_page_content', 'add_archive_URL_CPT1' );


function add_archive_URL_CPT2() {
	$post_type = 'my_custom_post_type_2';

	(REST OF THE FUNCTION)
}
add_filter( 'wpseo_sitemap_page_content', 'add_archive_URL_CPT2' );


function add_archive_URL_CPT3() {
	$post_type = 'my_custom_post_type_3';

	(REST OF THE FUNCTION)
}
add_filter( 'wpseo_sitemap_page_content', 'add_archive_URL_CPT3' );

[Al5ki]

Creating multiple functions didn't' work for me. It only took notice of the last one (despite varying IDs).

In the end I created Pages with the same Slug as the Archive Pages and they appeared in the index but organically redirected to the correct page.

[joelnewcomer]

I needed a solution for multiple languages in WPML so I've modified this code for that and am sharing in case anyone else needs it (I'm not sure why the insert code feature isn't wanting to work for me):

`
function add_archive_URL() {
$post_type = 'my_custom_post_type';
$url = '';

$active_languages = apply_filters( 'wpml_active_languages', false );
$current_language = apply_filters( 'wpml_current_language', NULL );

if ( $active_languages ) {
	foreach ( $active_languages as $code => $data ) {
    	do_action( 'wpml_switch_language', $code ); 

		$archive_url = get_post_type_archive_link( $post_type );
		
		$args = [
			'posts_per_page'         => 1,
			'post_type'              => $post_type,
			'orderby'                => 'modified',
			'order'                  => 'DESC',
			'no_found_rows'          => true,
			'update_post_meta_cache' => false,
			'update_post_term_cache' => false,
			'fields'                 => 'ids',
			'cache_results'          => true
		];
		
		$latest = new WP_Query( $args );
		
		while ( $latest->have_posts() ) {
			$latest->the_post();
			$date = get_post_modified_time( 'Y-m-d h:i:s', true );
			$last_mod = YoastSEO()->helpers->date->format( $date );
		}
		
		$url .= "\t<url>\n";
		$url .= "\t\t<loc>$archive_url</loc>\n";
		$url .= "\t\t<lastmod>$last_mod</lastmod>\n";
		$url .= "\t</url>\n";
    	
    }
    do_action( 'wpml_switch_language', $current_language );
}	
return $url;

}
add_filter( 'wpseo_sitemap_page_content', 'add_archive_URL' );`

[warudin]

"Show [CPT-name] in search results?" -> Off
"Show the archive for CPT-name in search results?" -> On

With the above settings, it would be expected to see the archive page in the sitemap. This is currently not the case.

I can see the 'robots' meta tag in the head of the archive page change from 'index' to 'noindex' when changing the second setting, fortunately.