One GET of every URL before it is sent to Yandex, Bing and the other IndexNow engines. A
page that says noindex, that robots.txt disallows, that names another canonical URL, that redirects, or that the
origin does not serve, is not submitted; each becomes a skipped Result with a stable Reason
(noindex, robots_disallowed, non_canonical, redirected, origin_error) in the log, the listeners, the PSR-14
events and the submission store. The engines see only URLs worth crawling — announcing a noindex page or a redirect
is at best wasted quota and at worst a reason for an engine to trust the key less.
What it never blocks: 404 and 410. A URL that answers 404 or 410 is submitted as is — that is the deletion signal IndexNow accepts, and the reason to submit a removed page in the first place. IndexNow is a notification, not indexing: the engine decides whether and when to crawl.
Off by default. Installing the package changes nothing until verify.enabled: true.
Русская версия · Issues and pull requests: github.com/indexnowkit/php (the php-* repositories are read-only splits)
composer require indexnowkit/verify # brings indexnowkit/core; nothing elseWith a framework adapter (indexnowkit/symfony-bundle, laravel, yii2) that is all: the adapter finds the package,
adds the verify block to its configuration and, once verify.enabled is true, decorates its submitter — the
queue, Messenger and yii2-queue workers verify too, because they get the same submitter. check prints one line
about it either way (verify: installed, disabled (verify.enabled: false)).
# Symfony: config/packages/indexnowkit.yaml # Laravel: config/indexnow.php 'verify' => [...]
indexnowkit: # Yii2: 'verify' => [...] of the component
verify:
enabled: true
redirect: skip # skip | follow
non_canonical: skip # skip | replace
origin_error: skip # skip | send| The URL answers | Decision |
|---|---|
200 with X-Robots-Tag: noindex / none (no bot prefix, or an IndexNow engine's bot; googlebot: is another engine) or <meta name="robots" content="noindex"> in <head> |
skipped, Reason::Noindex |
200 and robots.txt of the host disallows the path for * or an engine's bot (checked before the GET) |
skipped, Reason::RobotsDisallowed |
200 with Link: <…>; rel="canonical" or <link rel="canonical"> naming another URL (after the core's normalization: tracking parameters, trailing slash) |
non_canonical: skip → skipped, Reason::NonCanonical; replace → the canonical is submitted instead, when its host is one of yours |
3xx |
redirect: skip → skipped, Reason::Redirected; follow → the chain is followed (max_redirects, http(s), your hosts only) and the target verified; after a 301/308 both URLs are submitted (the page moved), after a 302/303/307 only the original |
404, 410 |
submitted — a deletion |
401, 403, 5xx, any other 4xx, a timeout or connection failure |
origin_error: skip → skipped, Reason::OriginError (retryable: a queue may try again); send → submitted with a warning. A 401/403 on a protected environment means: set verify.enabled: false there |
The GET is a real GET (not HEAD: many origins answer HEAD differently), without following redirects, with
verify.timeout (5 s) and its own User-Agent (verify.user_agent). Only the <head> of the first 256 KiB is read
for the signals; a body that is not text/html / application/xhtml+xml contributes headers only. Comments,
attribute order and case do not matter.
robots.txt is fetched once per host and process and kept in the PSR-16 cache behind debounce.store
(verify.robots_cache_ttl, an hour). A robots.txt that cannot be fetched (not 200, not 404) blocks nothing: one
warning per host, every path is treated as allowed — an unreachable robots.txt is no reason to keep quiet.
Batches. A batch above verify.max_batch (100) URLs is sent unverified with one warning: that is the sitemap
command, where the site itself lists its URLs (indexnow:sitemap --no-verify says so explicitly). submit --dry-run
runs the pre-flight (a GET is harmless) so it shows what would be skipped.
With dispatch: sync the pre-flight GETs run inside the web request that changed the page: N URLs cost N round trips
to your own origin, and on a single-process development server (symfony server:start without workers, php artisan serve, php yii serve) a request to yourself hangs until the timeout. check warns about it (verify.dispatch).
Use dispatch: queue / messenger, or keep verify.enabled: false outside production.
verify.delay (seconds, at most 30) waits before the first GET of a batch, so a queue worker that runs right after
the commit does not fetch the page from a cache that still holds the old version; it applies only outside a web
request and is ignored with dispatch: sync.
bin/console indexnow:check --sample=https://www.example.com/blog/post-1 --sample-class='App\Entity\Post'
php artisan indexnow:check --sample=https://www.example.com/blog/post-1 --sample-class='App\Models\Post:42'
php yii indexnow/check --sample=https://www.example.com/blog/post-1,https://www.example.com/ # Yii2: comma-separated, no comma inside a URLEvery sample is fetched and reported as one line (verify sample https://…: HTTP 200, index, canonical: self, robots: allowed, code verify.sample, the host of the URL). --sample-class takes a class annotated with #[IndexNow]
(<FQCN> — up to three objects of it, <FQCN>:<id> — that one) and resolves their URLs the way the submission does. A
noindex, a disallow, another canonical, a redirect, a 4xx/5xx or an unreachable origin is a warning, never an
error: production may be unreachable from CI, and the report must not fail a deploy over it. Without the package
--sample is an error (check --sample needs indexnowkit/verify), check without samples prints a no sample given line (code
verify.sample).
use IndexNowKit\Config;
use IndexNowKit\Http\TransportFactory;
use IndexNowKit\IndexNowKit;
use IndexNowKit\Key\StaticKeyProvider;
use IndexNowKit\Url\UrlNormalizerFactory;
use IndexNowKit\Verify\VerifyConfig;
use IndexNowKit\Verify\VerifyingSubmitter;
$core = Config::fromEnv();
$verify = VerifyConfig::fromArray(['enabled' => true, 'redirect' => 'follow']);
$indexNow = IndexNowKit::create($core);
$submitter = new VerifyingSubmitter(
$indexNow->submitter,
TransportFactory::lazy($verify->transportConfig($core), null, ['User-Agent' => $verify->userAgent()]),
$verify,
StaticKeyProvider::fromConfig($core),
UrlNormalizerFactory::fromConfig($core),
);
$results = $submitter->submit(['https://www.example.com/blog/post-1']);VerifyingSubmitter implements SubmitterInterface; listeners registered on it see every result (the inner ones and
the skipped ones) exactly once; pass the same PSR-14 dispatcher and submission store you gave the inner submitter so
the skipped results reach them too. The log lines are fixed strings operators can grep
(docs/operations.md); the options are in docs/configuration.md.
PHP 8.2+, indexnowkit/core ^0.10; indexnowkit/console ^0.3 for the --sample options of the check command (every
adapter has it).
- Composer package
indexnowkit/verify(add-on ofindexnowkit/core). With a framework adapter nothing is wired by hand:verify.enabled: truein the adapter's configuration decorates the submitter; the workers of the queue verify too. - Minimal complete snippet (every
useincluded):
use IndexNowKit\Config;
use IndexNowKit\Http\TransportFactory;
use IndexNowKit\IndexNowKit;
use IndexNowKit\Key\StaticKeyProvider;
use IndexNowKit\Url\UrlNormalizerFactory;
use IndexNowKit\Verify\VerifyConfig;
use IndexNowKit\Verify\VerifyingSubmitter;
$core = Config::fromEnv();
$verify = VerifyConfig::fromArray(['enabled' => true]);
$indexNow = IndexNowKit::create($core);
$submitter = new VerifyingSubmitter($indexNow->submitter, TransportFactory::lazy($verify->transportConfig($core), null, ['User-Agent' => $verify->userAgent()]), $verify, StaticKeyProvider::fromConfig($core), UrlNormalizerFactory::fromConfig($core));
$submitter->submit(['https://www.example.com/page']); // skipped Results carry Reason::Noindex, RobotsDisallowed, NonCanonical, Redirected, OriginError- Verify:
bin/console indexnow:check --sample=https://www.example.com/page,php artisan indexnow:check --sample=…,php yii indexnow/check --sample=…— one line per sample, warnings only;checkalone prints theverify:line (installed / disabled / not installed) and, with the package, a "no sample given" line. - Pitfalls:
verify.enabledisfalseby default; installing changes nothing. Withdispatch: syncthe GETs run inside the web request (checkwarns): use a queue.- 404 and 410 pass on purpose (deletions);
verify.redirect: followsubmits both URLs after a 301/308, only the original after a 302/303/307. - A canonical or a redirect target on a host without a key in
hosts/base_urlis a skip, never a submission: a page cannot make you announce another site. - robots.txt unavailable (500, timeout) allows everything with one warning; 404 is the normal "no robots.txt".
- Batches above
verify.max_batch(100) go unverified with a warning — that is thesitemapcommand;indexnow:sitemap --no-verifysays it explicitly. indexnow:check --sampleneeds this package; without it the option is an error with the install line.dispatch: autoexists in Symfony and Yii2, not in Laravel; locales arerouter.locales(Laravel),router.languages(Yii2),framework.enabled_locales(Symfony).
SemVer; until 1.0 minor versions may contain breaking changes, listed in CHANGELOG.md. What the compatibility promise covers: docs/bc.md.
MIT. IndexNow is a trademark of its owner; this project is independent and not affiliated with Microsoft, Yandex or indexnow.org.