Skip to content
Permalink
Browse files

bs4.element.NavigableString is not Pickleable

Turns out that bs4 uses bs4.element.NavigableString objects for results of searches, which aren't navigable. Cast to unicode and hope for the best.
  • Loading branch information
om-henners committed Jun 26, 2014
1 parent e03c150 commit 05311eb158428c74447a848e9465cb7d4fdb8479
Showing with 3 additions and 3 deletions.
  1. +3 −3 scraper.py
@@ -20,10 +20,10 @@
for user in user_details:
temp_name = user.find(name='a', attrs={"href" : re.compile("/users/")})
data = {}
data['name'] = temp_name.string
data['name'] = unicode(temp_name.string)
temp_location = user.find("span", "user-location")
data['location'] = temp_location.string
print data
data['location'] = unicode(temp_location.string)
#print data
scraperwiki.sqlite.save(unique_keys=['name'], data=data) #saving the data

i += 1

0 comments on commit 05311eb

Please sign in to comment.
You can’t perform that action at this time.