A basic harvester for Sina Weibo public post data as part of Social Feed Manager.
Provides harvesters for Sina Weibo API and Html Parser
Harvesting is performed by Weibowarc.
##Harvesting through friends_timeline Since the limited of API calling, the basic harvester uses the friendship to collecting the data.
##Getting a full list of followers Usually, It can call the friends API to get the full list of following friends.However, the API calling only results 30% of them. Currently, a full list of followers can get through the html parser analyzing the simple mobile website contents, since it hasn't decorated with css or javascript.
##Search key words Since the same reason of limitation of API calling, searing key word also rely on parsing the search result pages, but we can't get the exact time of the post. It can return the user name and text of the weibo post.
The html parser is just a possible solution for any exist limitation problem. If possible, the weibo harvester will rely most on the API calling.