Skip to content
/ Crawler Public
forked from gontarini/Crawler

Software to gather specific nodes in social services like: facebook, twitter, youtube

Notifications You must be signed in to change notification settings

engsbk/Crawler

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

**CRAWLER APPLICATION**

Application aim is to collect all interested data from each social media like: Facebook, Twitter and Youtube.

Facebook:

The process collect every each of existing page. It is started from sotrender liked pages and then exploring every single page liked and their liked pages and so on, and so on.

Youtube:

Application collect every existed channel. Whole algorithm explore more and more on base of featured channnels urls paramater, which indicates next channels. Rest of that looks the same as in case of facebook.

Twitter:

Application start its process from the given verified base by twitter itselfs and collects accounts seems to be companies or worth collected one (different case is for instance sports star, singer etc). To be qualified for datebase it must be performed certain conditions

Executing:

To run application it is neccessery specifing arguments. There are examples of executing application below:
$ python graph_retrieving.py facebook 'process_name'(without building new database)
$ python graph_retrieving.py facebook 'process_name' init (with building new one) - this command will start process for facebook and also construct database from scratch
You can change variable facebook to twitter or youtube
Algorithms are explained correctly in included folder UML.

REMEMBER TO EDIT CONFIGURATION FILE BEFORE EXECUTING!

About

Software to gather specific nodes in social services like: facebook, twitter, youtube

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%