simple-functional-robots-txt-parser

MIT

needed a simple robots.txt parser for https://github.com/franzenzenhofer/f19n-obtrusive-livetest

with analysis output what matched and why

and with a simple function interface

the actually fetching of the robots.txt must be done with some other (fetching) logic

now it exists