Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Java Robot Exclusion Compliance Library
Java Shell Groovy
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
gradle/wrapper
src
.gitignore
LICENSE.txt
README.md
build.gradle
gradlew
gradlew.bat
settings.gradle

README.md

JRobotX

Library to provide compliance with the Web Robot Exclusion protocol (robots.txt)

Usage

import com.trigonic.jrobotx.RobotExclusion;

// ...

RobotExclusion robotExclusion = new RobotExclusion();
if (robotExclusion.allows(url, userAgentString)) {
    // do something with url
}
Something went wrong with that request. Please try again.