This is a simple package for parsing robots.txt files. Based on http://www.robotstxt.org/orig.html
forked from suranyami/robots
mwotton/robots
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- DIGITAL Command Language 78.6%
- Haskell 8.0%
- Ruby 6.8%
- Perl 3.4%
- JavaScript 3.0%
- xBase 0.1%
- Other 0.1%