Skip to content

Commit

Permalink
Update scraper.py
Browse files Browse the repository at this point in the history
  • Loading branch information
acvwebb committed Jan 25, 2018
1 parent 693f801 commit 07f1139
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion scraper.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@
# # Find something on the page using css selectors
root = lxml.html.fromstring(html)
print root
print root.cssselect("a")
print root.cssselect("div[align='left']")
selectstuff = root.cssselect("a")
for i in selectstuff:
print i.text
print i.attrib['href']

urltoscrape = "http://site.com/"
Listylist = ["p1","p2","p3"]
Expand All @@ -27,6 +31,7 @@
fullurl = urltoscrape + blah
print fullurl


#
# # Write out to the sqlite database using scraperwiki library
# scraperwiki.sqlite.save(unique_keys=['name'], data={"name": "susan", "occupation": "software developer"})
Expand Down

0 comments on commit 07f1139

Please sign in to comment.