Generator robots.txt
-
Updated
Sep 8, 2017 - PHP
Generator robots.txt
Kirby 2 CMS plugin that adds a route for robots.txt
Manage robots.txt and sitemap.xml via the WordPress admin
Robots for Kirby CMS
WordPress plugin that warns you of server IP address changes, so you can take care of Search Engine Visibility settings.
Declarative, scriptable web robot (crawler) and scrapper
🔧 Robots.txt generator component for Nette framework.
User-Agent parser for robots.txt, X-Robots-tag and Robots-meta-tag rule sets
An extensible robots.txt parser and client library, with full support for every directive and specification.
Middleware to avoid search engine indexing with PSR-7 using robots.txt and X-Robots-Tag
Provides various framework-agnostic ways to generate the contents of the robots.txt file
PSR-15 middleware to enable/disable the robots of the search engines
A package for generating a robots.txt programmatically.
This Laravel Nova Tool gives your admins the ability to edit the robots.txt file from within the Nova control panel.
This is ready to use template to quickly start selling domain with minimum setup.
Behat extension for testing some On-Page SEO factors: meta title/description, canonical, hreflang, meta robots, robots.txt, redirects, sitemap validation, HTML validation, performance...
Kirby 3 CMS plugin that adds a route for robots.txt
A Simple Robots.txt Manager Plugin For Wordpress.
Add a description, image, and links to the robots-txt topic page so that developers can more easily learn about it.
To associate your repository with the robots-txt topic, visit your repo's landing page and select "manage topics."