Public, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores
Switch branches/tags
Nothing to show
Clone or download
Latest commit 74f5cbe Dec 9, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
abercrombie.com Fixed pagination for Abercrombie & Fitch scraper Feb 8, 2018
adidas.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
adidashardware.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
ae.com Fixed AE config Dec 9, 2018
alexandermcqueen.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
americanapparel.net Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
anntaylor.com Fixed JS parsing for Ann Taylor scraper Feb 8, 2018
anthropologie.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
asos.com Merge branch 'master' of https://github.com/Diggernaut/configs Jul 4, 2018
athleta.gap.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
avito.ru Added 1 new scraper Jan 19, 2018
bananarepublic.gap.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
bedbathandbeyond.com Data for new version of config Nov 21, 2018
benmeadows.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
betseyjohnson.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
bloomingdales.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
burton.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
cabelas.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
calypsostbarth.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
cartier.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
chanel.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
covergirl.com Added 5 new scrapers and updated sample CSV datasets Dec 27, 2017
debshops.com Added 5 new scrapers Jan 11, 2018
desigual.com Added 5 new scrapers Jan 11, 2018
diesel.com Added 5 new scrapers Jan 11, 2018
dior.com Added 5 new scrapers Jan 11, 2018
donnakaran.com Added 5 new scrapers Jan 11, 2018
dwell.co.uk Added 5 new scrapers Jan 12, 2018
eddiebauer.com Added 5 new scrapers Jan 12, 2018
eloquii.com Added 5 new scrapers Jan 12, 2018
esteelauder.com Added 5 new scrapers Jan 12, 2018
faithfullthebrand.com Added 5 new scrapers Jan 12, 2018
farfetch.com Added 12 new scrapers Jan 18, 2018
ferragamo.com Added 12 new scrapers Jan 18, 2018
footlocker.com Fixed footlocker.com config Mar 20, 2018
forever21.com Added 12 new scrapers Jan 18, 2018
freepeople.com Added 12 new scrapers Jan 18, 2018
gap.com Added 12 new scrapers Jan 18, 2018
gapfactory.com Added 12 new scrapers Jan 18, 2018
gentrynyc.com Added 12 new scrapers Jan 18, 2018
gilt.com Added 12 new scrapers Jan 18, 2018
guess.com Added 12 new scrapers Jan 18, 2018
guessbymarchiano.guess.com Added 12 new scrapers Jan 18, 2018
happysocks.com Added 6 new scrapers Jan 19, 2018
hermanmiller.com Added 6 new scrapers Jan 19, 2018
hermes.com Added 6 new scrapers Jan 19, 2018
hm.com Added 6 new scrapers Jan 19, 2018
hollisterco.com Added 6 new scrapers Jan 19, 2018
hugoboss.com Added 6 new scrapers Jan 19, 2018
ikea.com Added 6 new scrapers Feb 7, 2018
jaanuu.com Added 6 new scrapers Feb 7, 2018
jcrew.com Added 6 new scrapers Feb 7, 2018
joesjeans.com Added 6 new scrapers Feb 7, 2018
johnstonmurphy.com Added 6 new scrapers Feb 7, 2018
johnvarvatos.com Added 6 new scrapers Feb 7, 2018
usa.tommy.com Added 12 new scrapers Jan 18, 2018
vans.com vans.com config added Nov 26, 2018
.gitignore vans.com config added Nov 26, 2018
LICENSE Initial commit Dec 4, 2017
README.md Added 6 new scrapers Feb 7, 2018

README.md

configs

Public, free to use, repository with diggers configs for web scraping various e-commerce websites and online stores

All configs can be used with Diggernaut service to retrieve products information.

  1. You need to create free account at Diggernaut
  2. Login to your account
  3. Create a project with any name and description you want
  4. Get into your new project by clicking it and create new digger with any name
  5. Then you will see 3 options suggested to you, you need to use one where you will use meta-language
  6. Config editor will open and you can simply copy and paste config code and click on save button.
  7. Switch mode for digger from Debug to Active and then run your digger.
  8. Wait for completion.
  9. Download data.
  10. Schedule your runs if required.

List of sites:

  1. abercrombie.com
  2. adidas.com
  3. adidashardware.com
  4. ae.com
  5. alexandermcqueen.com
  6. americanapparel.net
  7. anntaylor.com
  8. anthropologie.com
  9. asos.com
  10. athleta.gap.com
  11. avito.ru
  12. bananarepublic.gap.com
  13. bedbathandbeyond.com
  14. benmeadows.com
  15. betseyjohnson.com
  16. bloomingdales.com
  17. burton.com
  18. cabelas.com
  19. calypsostbarth.com
  20. cartier.com
  21. chanel.com
  22. covergirl.com
  23. debshops.com
  24. desigual.com
  25. diesel.com
  26. dior.com
  27. donnakaran.com
  28. dwell.co.uk
  29. eddiebauer.com
  30. eloquii.com
  31. esteelauder.com
  32. faithfullthebrand.com
  33. farfetch.com
  34. ferragamo.com
  35. footlocker.com
  36. forever21.com
  37. freepeople.com
  38. gap.com
  39. gapfactory.com
  40. gentrynyc.com
  41. gilt.com
  42. global.tommy.com
  43. guess.com
  44. guessbymarciano.guess.com
  45. happysocks.com
  46. hermanmiller.com
  47. hermes.com
  48. hm.com
  49. hollisterco.com
  50. hugoboss.com
  51. ikea.com
  52. jaanuu.com
  53. jcrew.com
  54. joesjeans.com
  55. johnstonmurphy.com
  56. johnvarvatos.com

To be released:

  1. karenwalker.com
  2. karl.com
  3. katespade.com
  4. kennethcole.com
  5. kohls.com
  6. lancome-usa.com
  7. levi.com
  8. llbean.com
  9. lornajane.de
  10. lululemon.com
  11. macys.com
  12. mango.com
  13. marcjacobs.com
  14. marketplace.asos.com
  15. matthewwilliamson.com
  16. maybelline.com
  17. michaelkors.com
  18. nastygal.com
  19. nautica.com
  20. neimanmarcus.com
  21. nike.com
  22. nordstrom.com
  23. oldnavi.gap.com
  24. onekingslane.com
  25. patagonia.com
  26. paulsmith.com
  27. pier1.com
  28. potterybarn.com
  29. quiksilver.com
  30. rainbowshops.com
  31. ralphlauren.com
  32. ralphlaurenhome.com
  33. rei.com
  34. revlon.com
  35. revolve.com
  36. riverisland.com
  37. romwe.com
  38. roomandboard.com
  39. rue21.com
  40. scotch-soda.com
  41. sephora.com
  42. shopbop.com
  43. eastdane.com
  44. sonos.com
  45. stellamccartney.com
  46. stevemadden.com
  47. style.com
  48. sunglasshut.com
  49. superdry.com
  50. surlatable.com
  51. target.com
  52. tedbaker.com
  53. timberland.com
  54. tomford.com
  55. topman.com
  56. topshop.com
  57. toysrus.com
  58. uk.tommy.com
  59. underarmour.com
  60. uniqlo.com
  61. urbanoutfitters.com
  62. us.benetton.com
  63. us.burberry.com
  64. us.louisvuitton.com
  65. us.nextdirect.com
  66. us.shein.com
  67. valentino.com
  68. vans.com
  69. versace.com
  70. victoriassecret.com
  71. vineyardvines.com
  72. walmart.com
  73. westelm.com
  74. whitehouseblackmarket.com
  75. williams-sonoma.com
  76. ysl.com
  77. zalando.de
  78. zappos.com
  79. zara.com
  80. zarahome.com