Skip to content

Commit

Permalink
Update scraper.py
Browse files Browse the repository at this point in the history
  • Loading branch information
yngveny committed Sep 25, 2017
1 parent 459120e commit 757e1c7
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion scraper.py
Expand Up @@ -6,12 +6,15 @@
# create a new function, which gets passed a variable we're going to call 'url'
def scrape_dof(url):
html = scraperwiki.scrape(url)

#print html
root = lxml.html.fromstring(html)
print root.find_class("div.notice-search-item")

#line below selects all <div class="notice-search-item">
rows = root.cssselect("div.notice-search-item")
for row in rows:
print row.find_class("div.notice-search-item")
#print row.find_class("div.notice-search-item")
# Set up our data record - we'll need it later
record = {}
a = row.cssselect("a") #grab all <a> tags within our <div>
Expand Down

0 comments on commit 757e1c7

Please sign in to comment.