Skip to content

moshee/scrape

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Scrape

This is a collection of various website scrapers that I write eventually when I want to automate data acquisition. It will grow very slowly, perhaps not at all. I hope none of the websites targeted will mind. If they do, I will take that particular module/package offline.

Most of the scrapers will be written in Ruby. If I find that I need performance, I will rewrite them in Go. These rewritten ones may go in a different repo. This will be unlikely as the pauses between requests usually outweigh computation time by a large factor. If I find necessary, I may one day replace hpricot with nokogiri.

I am experimenting with packaging gems as well. It appears to function correctly as a gem.

License is inherited from dependencies where applicable.

Documentation is YARD-style. I can't guarantee its quality, though.

About

Some website scrapers

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages