-
Notifications
You must be signed in to change notification settings - Fork 0
anupdhml/url_inspect
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
By anupdhml Dependenceies: bs4 (beautiful soup4), lxml. Install from pip/easy_install OR the SRC folder (has bs4) A module meant to emulate google's preview (or 'inspect') feature for search results (see file google_feature.png) # make_tables.py ############################################################# usage: python make_tables.py <url or html file> python make_tables.py -f <file with list of urls or html file paths> Extract all the link sets from urls/html provided, and if used in batch mode (-f option), generate a file with the table of all the link labels, arranged by their frequency over all the pages. The link labels are stemmed using the porter stemming algorithm. # hij_inspect.py ############################################################# usage: python hij_inspect.py <table filename> <url or html file> Use the table generated by make_tables.py to calculate scores for each link set present in the specified url/html-file. The link set with the best score is shown in the search results, as a preview. Make sure that table file provided is appropriate for the used url/html file (eg: probably want news table for a site like cnn) # TESTING #################################################################### For testing, try these: python make_tables.py testfiles/serafina_mod.html python make_tables.py http://www.nytimes.com/ python make_tables.py -f testfiles/urls_test # file table_testfiles-urls_test is the result of the last command in the above list... python hij_inspect.py table_testfiles-urls_test http://www.cnn.com Sample output of these commands can be seen in the SAMPLE_RUN.txt file
About
Emulate link previews from google search
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published