Web crawler for checking the validity of your documents.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
data/schemas
doc
lib
man/man1
test
.gemtest
.gitignore
.rubocop.yml
.travis.yml
Gemfile
History.md
LICENSE
README.md
Rakefile
validate-website.gemspec
validate-website.png

README.md

validate-website

Description

Web crawler for checking the validity of your documents

validate website

Installation

Debian

apt install ruby-dev libxslt1-dev libxml2-dev

If you want complete local validation look tidy packages

RubyGems

gem install validate-website

Synopsis

validate-website [OPTIONS]
validate-website-static [OPTIONS]

Examples

validate-website -v -s https://www.ruby-lang.org/
validate-website -v -x tidy -s https://www.ruby-lang.org/
validate-website -v -x nu -s https://www.ruby-lang.org/
validate-website -h

Description

validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls (more info doc/validate-website.adoc).

validate-website-static checks the markup validity of your local documents with XML Schema / DTD (more info doc/validate-website-static.adoc).

HTML5 support with libtidy5 or Validator.nu Web Service.

Exit status

  • 0: Markup is valid and no 404 found.
  • 64: Not valid markup found.
  • 65: There are pages not found.
  • 66: There are not valid markup and pages not found.

On your application

require 'validate_website/validator'
body = '<!DOCTYPE html><html></html>'
v = ValidateWebsite::Validator.new(Nokogiri::HTML(body), body)
v.valid? # => false

Jekyll static site validation

You can add this Rake task to validate a jekyll site:

desc 'validate _site with validate website'
task validate: :build do
  Dir.chdir('_site') do
    sh("validate-website-static --site '<CONFIG_URL>'") do |ok, res|
      unless ok
        puts "validate error (status = #{res.exitstatus})"
      end
    end
  end
end

More info

HTML5

Tidy5

If the libtidy5 is found on your system this will be the default to validate your html5 document. This does not depend on a tier service everything is done locally.

Validator.nu web service

When --html5-validator nu option is used HTML5 support is done by using the Validator.nu Web Service, so the content of your webpage is logged by a tier. It's not the case for other validation because validate-website use the XML Schema or DTD stored on the data/ directory.

Please read http://about.validator.nu/#tos for more info on the HTML5 validation service.

Use validator standalone web server locally

You can download validator jar and start it with:

java -cp PATH_TO/vnu.jar nu.validator.servlet.Main 8888

Then you can use validate-website option:

--html5-validator-service-url http://localhost:8888/
# or
export VALIDATOR_NU_URL="http://localhost:8888/"

This will prevent you to be blacklisted from validator webservice.

Tests

With standard environment:

bundle exec rake

Credits

  • Thanks tenderlove for Nokogiri, this tool is inspired from markup_validity.
  • And Chris Kite for Anemone web-spider framework and postmodern for Spidr.

Contributors

See GitHub.

License

The MIT License

Copyright (c) 2009-2018 Laurent Arnoud laurent@spkdev.net


Build Version Documentation License Coverage Status Inline docs