Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 108 lines (76 sloc) 4.103 kB
e0de6bb @mikejs sphinx docs
mikejs authored
1 =======================
2 Overview
3 =======================
4
986b264 @rshapiro quick fix to url in README
rshapiro authored
5 If you are reading this page on github, please go `here <http://samburg.sunlightlabs.org/docs>`_ to read more complete documentation.
abd0ee8 @rshapiro added a link to documentation on samburg in main page
rshapiro authored
6
7d83ba4 @jamesturk updated readme
jamesturk authored
7 About the Project
8 =================
0827d36 @jamesturk first commit
jamesturk authored
9
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
10 The goal of the Fifty State Project is to build scrapers and parsers in order to get as much state
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
11 legislative data as possible in one place.
0827d36 @jamesturk first commit
jamesturk authored
12
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
13 For details on the reasons for the project and goals behind the project see
14 `the project announcement <http://sunlightlabs.com/blog/2009/02/26/fifty-state-project/>`_.
7d83ba4 @jamesturk updated readme
jamesturk authored
15
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
16 To stay up to date and communicate with other contributors to the project visit the `Fifty State Project Google Group <http://groups.google.com/group/fifty-state-project>`_.
17
7d83ba4 @jamesturk updated readme
jamesturk authored
18 For an editable overview of each state's progress visit the `Sunlight Labs Wiki <http://wiki.sunlightlabs.com/index.php/State_Legislation_Page>`_.
19
20 Project Goals
21 -------------
22
23 1. Collect URLs of State Legislature and Legislative Information Pages [done]
24 2. Grab legislators and legislation
25 1. Build scrapers and obtain data files for legislation in each of the fifty states
26 2. create sponsor relationship between legislators and legislation
27 3. Grab votes
28 1. Build scrapers and obtain data files for legislator votes on legislation
29 2. create voting relationship between legislators and legislation
30 4. Build tools on top of data
31
32
b30e30f updated documentation (still in progress)
bx authored
33 .. _usage:
34
3c9d3f3 @jamesturk explain what the workflow may look like once things are written
jamesturk authored
35 Usage (proposed)
36 ----------------
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
37 Valid options:
2debaf8 @mikejs ruby implementation of new scraper format, docs
mikejs authored
38 * ``--year``: a year or years the parser should attempt
39 * ``--all``: Attempt to parse years from 1969-2009
40 * ``--upper``: Parse upper chamber
41 * ``--lower``: Parse lower chamber
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
42
43 The vision is that the flow will look something like this:
44 $ ./scripts/nc/get_legislation --year=2009 --upper
3c9d3f3 @jamesturk explain what the workflow may look like once things are written
jamesturk authored
45
46
7d83ba4 @jamesturk updated readme
jamesturk authored
47 Contributing
48 ============
49
b30e30f updated documentation (still in progress)
bx authored
50 If you are interested in contributing the recommended procedure is to
51 check on the `Sunlight Labs Wiki
52 <http://wiki.sunlightlabs.com/index.php/Fifty_State_Project#Status>`_
53 and in the repository to see where your state is. The next step is to
54 announce your interest on the `Fifty State Project Google Group
55 <http://groups.google.com/group/fifty-state-project>`_ (this is where
56 you can ask questions and make suggestions regarding the project).
7d83ba4 @jamesturk updated readme
jamesturk authored
57
58 Managing a State
59 ----------------
60
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
61 Once you have claimed a state on the wiki and mailing list you should probably
62 maintain your own fork of the project on `github <http://github.com>`_.
7d83ba4 @jamesturk updated readme
jamesturk authored
63
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
64 Please avoid making changes to files in other states/etc. on your state branch.
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
65 Stick to editing files in the scripts/*your_state* directory and where necessary
66 in any relevant utils directories.
7d83ba4 @jamesturk updated readme
jamesturk authored
67
d0a9456 @Caged more documentation updates. document how to use rbutils
Caged authored
68 Whenever your state script works as it should announce it on the mailing list and
69 someone will merge your changes into the core.
7d83ba4 @jamesturk updated readme
jamesturk authored
70
208c170 began docs on contributing/python
bx authored
71 .. _licensing:
72
7d83ba4 @jamesturk updated readme
jamesturk authored
73 Licensing
74 ---------
75
bbabba9 @jamesturk fixed link to GPLv3
jamesturk authored
76 As of June 15th 2009 the Fifty State Project is licensed under the `GPLv3 license <http://www.fsf.org/licensing/licenses/gpl-3.0.html>`_
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
77
2debaf8 @mikejs ruby implementation of new scraper format, docs
mikejs authored
78 See LICENSING for the full terms of the GPLv3.
7d83ba4 @jamesturk updated readme
jamesturk authored
79
80 Requirements
81 ============
82
2140b02 @rshapiro updated documentation
rshapiro authored
83 Although we have previously allowed you to write parsers in your
84 language of choice, for the sake of maintenance we *highly* encourage
85 you to write your parsers in Python. Currently Python is the only
86 language we are supporting with our documentation and tools. If you
87 would like to contribute in a language other than Python, please send
88 an email to `Fifty State Project Google Group
89 <http://groups.google.com/group/fifty-state-project>`_ so we can
90 discuss the issue.
91
92 For details on how scripts should be written and how they should run see :doc:`scripts/pyutils/README`.
93
94 If you are completely unfamiliar with Python there is other things you
95 can do to help with the government transparency movement. Ruby developers are encouraged to work on the `Congrelate <http://congrelate.com>`_ Project. For other project ideas please join the `Sunlight Labs Google Group <http://groups.google.com/group/sunlightlabs>`_.
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
96
97
2140b02 @rshapiro updated documentation
rshapiro authored
98 .. _dependencies:
99
100 Dependencies
7d83ba4 @jamesturk updated readme
jamesturk authored
101 -------------
2140b02 @rshapiro updated documentation
rshapiro authored
102 * Python (2.5+)
7d83ba4 @jamesturk updated readme
jamesturk authored
103 * BeautifulSoup
c8c83cf @mikejs some docs
mikejs authored
104 * html5lib
105 * simplejson if on Python 2.5
86dec1a @jamesturk officially convert license to GPLv3 and clarify a few things in README
jamesturk authored
106 * (this list is out of date, refer to specific scripts/state directories for dependencies)
7d83ba4 @jamesturk updated readme
jamesturk authored
107
Something went wrong with that request. Please try again.