wlz1028/pl_works
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
Public DNS/IP: http://ec2-54-174-36-73.compute-1.amazonaws.com/search 54.174.36.73 Easier access on ur phone http://bit.ly/hitsearch How to deploy a new website: 1) remove csc326_group18.pem from backEnd/ if you see one 2) remove csc326_group18 key pairs from AWS console if you see one 3) > cd backEnd > python one_key_deployment.py ACCESS_KEY SECRETE_KEY Note: use port 80 instead of 8080 Pound will redirect port 80 traffic to two bottle apps on 8081 and 8082 How to terminate a instance: > cd backEnd > python one_key_terminate.py ACCESS_KEY SECRETE_KEY INSTANCE_ID Final report: rpts/group18_csc3262014.pdf Code structure: - backEnd/ #backEnd source code - deploy.sh #deploy.sh auto run on aws - frontEnd/ #frontEnd source code - rpts/ #reports frontEnd/ frontEnd.py #bottle application mongodb.py #query mongodb and mongodb api word_correction.py #word_correction api dictionary.txt #auto correction dictionary *tpl #bottle templates *css #css files backEnd/ crawler.py #single thread crawler crawler_multi_Thread.py #multithread cralwer unit_test.py #tester for cralwer and mongodb master.py #auto crawl and build mongodb one_key_deployment.py #launch new aws instance and deploy website one_key_terminate.py #termiate a instance by id pagerank.py #pagerank api test.html #test html for cralwer urls.txt #list of urls to be crawl write_mongo.py #mongodb insertion api #Please see report for more details pagerank_mapreduce/ #input data files cbc_pr.txt pagerank.txt #mapper and reducer pg_mapper.py pg_mapper_generator.py #bash script to iterate mapreuce MR_iteration.sh #imprived mapper and reducer pg_reducer.py pg_reducer_generator.py