robogen

Robots.txt generator done right.

Takes an array of objects that describe rules. Converts them to an appropriate robots.txt.

Usage

Generate a robots.txt:

import robogen from 'robogen'

const robotsTXT = robogen([{
	userAgent: '*',
	allow: '/',
	disallow: ['/secret', '/private']
}, {
	userAgent: ['Bingbot', 'MSNBot'],
	disallow: '/'
}])

console.log(robotsTXT)

This will output

User-agent: *
Allow: /
Disallow: /secret
Disallow: /private
User-agent: Bingbot
User-agent: MSNBot
Disallow: /

Install

With npm installed, run

$ npm install robogen

Acknowledgments

robogen was inspired by robots-generator

See Also

• noffle/common-readme

License

MIT