Skip to content

wlz1028/pl_works

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

90 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Public DNS/IP:
  http://ec2-54-174-36-73.compute-1.amazonaws.com/search
  54.174.36.73
  Easier access on ur phone
  http://bit.ly/hitsearch

How to deploy a new website:
1) remove csc326_group18.pem from backEnd/ if you see one
2) remove csc326_group18 key pairs from AWS console if you see one
3)
  > cd backEnd
  > python one_key_deployment.py ACCESS_KEY SECRETE_KEY
Note: use port 80 instead of 8080
Pound will redirect port 80 traffic to two bottle apps on 8081 and 8082

How to terminate a instance:
> cd backEnd
> python one_key_terminate.py  ACCESS_KEY SECRETE_KEY INSTANCE_ID

Final report:
rpts/group18_­csc326­2014.pdf

Code structure:
  - backEnd/    #backEnd source code
  - deploy.sh   #deploy.sh auto run on aws
  - frontEnd/   #frontEnd source code
  - rpts/       #reports

  frontEnd/
      frontEnd.py #bottle application
      mongodb.py  #query mongodb and mongodb api
      word_correction.py #word_correction api
      dictionary.txt #auto correction dictionary
      *tpl #bottle templates
      *css #css files

  backEnd/
      crawler.py  #single thread crawler
      crawler_multi_Thread.py #multithread cralwer
      unit_test.py #tester for cralwer and mongodb
      master.py #auto crawl and build mongodb
      one_key_deployment.py #launch new aws instance and deploy website
      one_key_terminate.py  #termiate a instance by id
      pagerank.py #pagerank api
      test.html #test html for cralwer
      urls.txt  #list of urls to be crawl
      write_mongo.py  #mongodb insertion api

  #Please see report for more details
  pagerank_mapreduce/
      #input data files
      cbc_pr.txt
      pagerank.txt

      #mapper and reducer
      pg_mapper.py
      pg_mapper_generator.py
      #bash script to iterate mapreuce
      MR_iteration.sh

      #imprived mapper and reducer
      pg_reducer.py
      pg_reducer_generator.py

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors