Skip to content

dansari2020/web-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Run Web Crawler on Luma site

ruby luma/main.rb

Export products

Export html
ruby luma/main.rb -e html
Export csv
ruby luma/main.rb -e csv
Export json
ruby luma/main.rb -e json

Sort products

Sort by name
ruby luma/main.rb -s name -t asc
Sort by Highest Price
ruby luma/main.rb -s price -t desc

Log

show log and create file in path logs/YYYY-mm-dd.txt
ruby luma/main.rb -l

Deep Page

Fetch number of pages
Fetch just 5 pages
ruby luma/main.rb -p 5

Sample

ruby luma/main.rb -l -p 10 -e html -s price -t desc 

About

Web Crawler is the automated fetching of all products of a web pages by a software process.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published