Skip to content
Java Robot Exclusion Compliance Library
Java Shell Groovy
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
gradle/wrapper
src
.gitignore
LICENSE.txt
README.md
build.gradle
gradlew
gradlew.bat
settings.gradle

README.md

JRobotX

Library to provide compliance with the Web Robot Exclusion protocol (robots.txt)

Usage

import com.trigonic.jrobotx.RobotExclusion;

// ...

RobotExclusion robotExclusion = new RobotExclusion();
if (robotExclusion.allows(url, userAgentString)) {
    // do something with url
}
Something went wrong with that request. Please try again.