Skip to content

tomtt/wikileaks_wardiary_on_rails

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

== About wikileaks_warlogs
This is an example rails app with the migrations to import the raw wikileaks war-log data and some simple views to display the data.

Github: http://github.com/tomtt/wikileaks_wardiary_on_rails

You can find a working release of this app at http://war-logs-on-rails.com

== Install
* For instructions on how to get Ruby/Rails: http://rubyonrails.org/download
* The data will be imported by the migrations from a file compressed using 7za. This command needs to work
  on your system to be able to perform the import. Please get it from http://www.7-zip.org/download.html.
* create your database.yml file, using one of the config/database.yml.*.example files if you prefer
* rake db:create:all
* rake:db:migrate (create tables, imports the orignal data and makes it 'railsy')
* rake db:seed (this loads some of the metadata)
* rake wardiary:term_definitions:tag_war_events (this tags all the war events with the terms currently
  defined in the system. Beware: it can take a long time (hours) to run this task)

The import will currently not work on systems that do not support 'piping' data to another process. If you
can think of an efficient way to import the compressed data on these systems (i.e. windows) please submit
a patch.

== What's next?
I'm open to anybody helping out, let me know if you plan to work on something so that we will not have multiple people working on the same thing.

Some ideas of things that could be done next:
* Whatever you think will best help people find things they are looking for in the data
* Whatever you think will best help people build up a mental picture of what occured for an event
* Whatever you think will cause more people to look at this
* Suggestions:
  * Create WarEventView model that counts how many times an event has been looked at
  * Normalize data and ability to filter based on this (type_of_event, category, region, attack_on, reporting_unit, unit_name, type_of_unit, affiliation)
  * Adding a TermDefinition or changing it pattern should automatically update its taggings. This is expensive, so there also needs to be a flag to turn updating on off, when the flag is off and set to on, it should do the update then and from then on.
  * Crowd sourcing (could maybe be a seperate app?)
    * Allow people to tag any cable with any tag
    * Allow people to flag a cable as interesting/non interesting
    * Allow people to find cables related to a news article they know about, i.e.: the ability to narrow the potential cables down based on information provided by news articles (date, location, ?)
    * Allow people to submit stories/additional info/images/video related to a cable
    * Allow people to relate one cable to another with a reason why they are related
    * I18n: allow people to translate the fields of a war_event
  * Client side js to highlight interesting info: e.g. in the table view, rows with kia/wia > 0 could have subtle highlighting with the cells themselves having a color which is darker for higher numbers
  * Geo based
    * Compute a grouping by region, ability to click on the map and see nearby events
    * Show a heatmap overlayed on the map for an index
    * Show clickable items on a map
    * Some fancy tool where you can pick certain criteria and see the matching events on the map (not sure if this is possible to do given the scale of the dataset)
    * Extract grid references from summary, translate to lat/lng, ability to search based on them
  * More interesting stats
    * For each tag: the number of cables it appears in
    * Cumulative kia/wia
  * Graphs for statistics (google chart?)
  * Parse out info on military facilities and link with other info like (http://www.globalsecurity.org/military/facility/afghanistan.htm)
  * Design & create a better layout
  * Provide a mouseover popup for known terms (http://wardiary.wikileaks.org/ already does this, maybe some js code can be borrowed)
  * A nicely designed mouseover on each index that shows summary and other key info
  * Fancier tag cloud
  * An api so that trusted parties can contribute information to the site
  * Ability to sort by the values in a column

== About me
* http://www.unboxedconsulting.com/people/tom-ten-thij
* war-logs-on-rails@tomtenthij.nl
* @tomtt
* http://github.com/tomtt
* http://workingwithrails.com/person/2527-tom-ten-thij

== Sources
* http://wardiary.wikileaks.org
* http://tpmcafe.talkingpointsmemo.com/talk/blogs/k/g/kgb999/2010/07/complete-wikileaks-afghan-war.php
* http://www.guardian.co.uk/world/series/afghanistan-the-war-logs

== Note on Patches/Pull Requests
 
* Fork the project.
* Make your feature addition or bug fix.
* Add tests for it. This is important so I don't break it in a
  future version unintentionally.
* Commit, do not mess with rakefile, version, or history.
  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
* Send me a pull request. Bonus points for topic branches.

== Copyright

Copyright (c) 2010 Tom ten Thij. See LICENSE for details.

About

A basic rails install with migrations for importing the original wikileaks wardiary data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published