Permalink
Browse files

Merge branch 'master' of github.com:neilkod/oow-vote-hacking

  • Loading branch information...
2 parents 1a7e2ad + 3cf156e commit 3e8ac1b0f4d2e832bffb78aa980b1fecad9d5427 @neilkod neilkod committed Jun 17, 2011
Showing with 6,097 additions and 4,972 deletions.
  1. +6 −0 0_make_directories.sh
  2. +11 −0 1_get_proposal_abstracts.sh
  3. +10 −0 2_get_proposals.sh
  4. +14 −0 3_extract_votes.sh
  5. +40 −0 4_create_flat_files.sh
  6. +5 −0 data/sessions.dat
  7. +225 −2 data/{voters.dat → users.dat}
  8. +5,759 −4,969 data/votes.dat
  9. +26 −0 load_data.sql
  10. +1 −1 oow.py
@@ -0,0 +1,6 @@
+#!/bin/bash
+
+mkdir proposal_abstracts
+mkdir proposals
+mkdir votes
+mkdir data
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+## as of June 17th there are 39 pages of proposals sorted by title
+## see https://mix.oracle.com/events/oow11/proposals to determine this value
+
+cd proposal_abstracts
+rm *.html
+for i in {1..39}
+do
+curl -o $i.html "https://mix.oracle.com/events/oow11/proposals?campaign_id=oow11&page=${i}&sort=title"
+done
View
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+## get the relative link to each individual proposal
+cd proposal_abstracts
+cat *.html | grep "<p class=\"title\">" | cut -d'"' -f4 > ../proposal_urls.txt
+
+## download each individual proposal page
+cd ../proposals
+rm *
+wget --no-check-certificate --base=https://mix.oracle.com/events/oow11/proposals -i ../proposal_urls.txt
View
@@ -0,0 +1,14 @@
+#!/bin/bash
+
+## clean up old vote extract
+cd votes
+rm *
+cd ..
+
+## extract the voters from each proposal and put them in a file 1 per proposal
+cd proposals
+
+ls -1 | while read f
+do
+grep voting $f | grep 0px | cut -d'"' -f2 | cut -d "/" -f6 > ../votes/$f.txt
+done
@@ -0,0 +1,40 @@
+#!/bin/bash
+
+## flat file for sessions
+## format session_id|"title"|user_id
+
+cd proposals
+ls -1 | while read f
+do
+sid=$(echo $f | cut -d"-" -f1)
+t=$(grep h2 $f | perl -n -e '/<h2>(.*?)<\/h2>/ && print "$1\n"')
+aid=$(grep presenter-type $f | cut -d'"' -f2| cut -d"/" -f6)
+echo "${sid}|\"${t}\"|${aid}"
+done > ../data/sessions.dat
+cd ..
+
+## flat file for users (assumed to be a unique list of all voters)
+## format user_id|"name"
+
+cd votes
+cat * | sort -n -u | while read f
+do
+id=$(echo $f|cut -d"-" -f1)
+n=$(echo $f|cut -d"-" -f2-|sed "s/-/ /g")
+echo "$id|\"$n\""
+done > ../data/users.dat
+cd ..
+
+## flat file for votes
+## format session_id|user_id
+
+cd votes
+ls -1 | while read f
+do
+s=$(echo $f |cut -d"-" -f1)
+cat $f | while read l
+do
+v=$(echo $l | cut -d"-" -f1)
+echo "$s|$v"
+done
+done > ../data/votes.dat
View
@@ -374,3 +374,8 @@
11259|"AWR Performance data mining"|138040
11260|"How EBS R12 makes possible to implement fully Automated Cloning!"|138040
11261|"Configure Highly Available and secure Fusion Middleware environment"|137580
+11262|"Oracle CRM On Demand: Creating Workflows to Increase Operational Efficiency"|138264
+11263|"Installation of E1 9.0 without Ser. Mgr. due to client’s security policies"|138320
+11264|"Cloud----the horizon of opportunities for JDE"|138320
+11265|"Oracle JDE - Best fit for property management company"|74381
+11266|"Self Tuned Databases"|138409
Oops, something went wrong. Retry.

0 comments on commit 3e8ac1b

Please sign in to comment.