Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 203 lines (187 sloc) 9.787 kb
03c0695 Updated CHANGES.txt
Steve Johnson authored
1 v0.3.2, 2012-02-21 -- ???
2 * Docs:
3 * 'Testing with mrjob' section in docs (includes #321)
4 * MRJobRuner.counters() included in docs (#321)
5 * terminate_idle_job_flows is spelled correctly in docs (#339)
6 * Running jobs:
7 * local mode:
8 * Allow non-string jobconf values again (this changed in v0.3.0)
9 * Don't split *.gz files (#333)
10 * emr mode:
11 * Spot instance support via ec2_*_instace_bid_price and renamed instace
12 type/number options (#219)
13 * ami_version option to allow switching between EMR AMIs (#306)
14 * 'Error while reading from input file' displays correct file (#358)
15 * python_bin used for bootstrap_python_packages instead of just 'python'
16 (#355)
17 * Pooling works with bootstrap_mrjob=False (#347)
18 * Pooling makes sure a job flow has space for the new job before joining
19 it (#324)
20 * Tools:
21 * create_job_flow no longer tries to use an option that does not exist
22 (#349)
23 * report_long_jobs tool alerts on jobs that have run for more than X hours
24 (#345)
25 * mrboss no longer spells stderr 'stsderr'
26 * terminate_idle_job_flows counts jobs with pending (but not running)
27 steps as idle (#365)
28 * terminate_idle_job_flows can terminate job flows near the end of a
29 billable hour (#319)
30 * audit_usage breaks down job flows by pool (#239)
31 * Various tools (e.g. audit_usage) get list of job flows correctly (#346)
d7c7674 Moved CHANGES changes to appropriate version
Stephen Johnson authored
32
5eba42b releasing this tomorrow
David Marin authored
33 v0.3.1, 2011-12-20 -- Nooooo there were bugs!
6573ec3 update CHANGES.txt and __version__ for release
David Marin authored
34 * Instance-type command-line arguments always override mrjob.conf (Issue #311)
35 * Fixed crash in mrjob.tools.emr.audit_usage (Issue #315)
36 * Tests now use unittest; python setup.py test now works (Issue #292)
37
a794f6e bump release date to tomorrow
David Marin authored
38 v0.3.0, 2011-12-07 -- Worth the wait
877e307 massive update to CHANGES.txt (fixes #246)
David Marin authored
39 * Configuration:
40 * Saner mrjob.conf locations (Issue #97):
41 * ~/.mrjob is deprecated in favor of ~/.mrjob.conf
42 * searching in PYTHONPATH is deprecated
5a9868d bump version number, and plan on releasing on Friday
David Marin authored
43 * MRJOB_CONF environment variable for custom paths
877e307 massive update to CHANGES.txt (fixes #246)
David Marin authored
44 * Defining Jobs (MRJob):
45 * Combiner support (Issue #74)
46 * *_init() and *_final() methods for mappers, combiners, and reducers
47 (Issue #124)
55802c1 added another new feature to CHANGES.txt
David Marin authored
48 * mapper/combiner/reducer methods no longer need to contain a yield
49 statement if they emit no data
877e307 massive update to CHANGES.txt (fixes #246)
David Marin authored
50 * Protocols:
51 * Protocols can be anything with read() and write() methods, and are
52 instances by default (Issue #229)
53 * Set protocols with the *_PROTOCOL attributes or by re-defining the
54 *_protocol() methods
55 * Built-in protocol classes cache the encoded and decoded value of the
56 last key for faster decoding during reducing (Issue #230)
57 * --*protocol switches and aliases are deprecated (Issue #106)
58 * Set Hadoop formats with HADOOP_*_FORMAT attributes or the hadoop_*_format()
59 methods (Issue #241)
60 * --hadoop-*-format switches are deprecated
61 * Hadoop formats can no longer be set from mrjob.conf
62 * Set jobconf with JOBCONF attribute or the jobconf() method (in addition
63 to --jobconf)
64 * Set Hadoop partitioner class with --partitioner, PARTITIONER, or
65 partitioner() (Issue #6)
66 * Custom option parsing (Issue #172)
67 * Use mrjob.compat.get_jobconf_value() to get jobconf values from environment
68 * Running jobs:
69 * All modes:
70 * All runners are Hadoop-version aware and use the correct jobconf and
71 combiner invocation styles (Issue #111)
72 * All types of URIs can be passed through to Hadoop (Issue #53)
73 * Speed up steps with no mapper by using cat (Issue #5)
74 * Stream compressed files with cat() method (Issue #17)
75 * hadoop_bin, python_bin, and ssh_bin can now all take switches (Issue #96)
76 * job_name_prefix option is gone (was deprecated)
77 * Better cleanup (Issue #10):
78 * Separate cleanup_on_failure option
79 * More granular cleanup options
80 * Cleaner handling of passthrough options (Issue #32)
81 * emr mode:
82 * job flow pooling (Issue #26)
83 * vastly improved log fetching via SSH (Issue #2)
84 * New tool: mrjob.tools.emr.fetch_logs
85 * default Hadoop version on EMR is 0.20 (was 0.18)
86 * ec2_instance_type option now only sets instance type for slave nodes
87 when there are multiple EC2 instances (Issue #66)
c6b70bd Moved mrboss to changes EMR section
Steve Johnson authored
88 * New tool: mrjob.tools.emr.mrboss for running commands on all nodes and
89 saving output locally
877e307 massive update to CHANGES.txt (fixes #246)
David Marin authored
90 * inline mode:
91 * Supports cmdenv (Issue #136)
18e856f fixed passing of options to inline mode (fixes #301)
David Marin authored
92 * Passthrough options can now affect steps list (Issue #301)
877e307 massive update to CHANGES.txt (fixes #246)
David Marin authored
93 * local mode:
94 * Runs 2 mappers and 2 reducers in parallel by default (Issue #228)
95 * Preliminary Hadoop simulation for some jobconf variables (Issue #86)
96 * Misc:
97 * boto 2.0+ is now required (Issue #92)
98 * Removed debian packaging (should be handled separately)
92927ba Make it explicit that this is v0.3.0, and get a head start on CHANGES.tx...
David Marin authored
99
b74ea0d add v0.2.8 to CHANGES.txt
David Marin authored
100 v0.2.8, 2011-09-07 -- Bugfixes and betas
101 * Fix log parsing crash dealing with timeout errors
e6a7d46 edits to v0.2.8 changelog
David Marin authored
102 * Make mr_travelling_salesman.py work with simplejson
b74ea0d add v0.2.8 to CHANGES.txt
David Marin authored
103 * Add emr_additional_info option, to support EMR beta features
e6a7d46 edits to v0.2.8 changelog
David Marin authored
104 * Remove debian packaging (should be handled separately)
92fca19 Updated changelog about tmp bucket location
Steve Johnson authored
105 * Fix crash when creating tmp bucket for job in us-east-1
b74ea0d add v0.2.8 to CHANGES.txt
David Marin authored
106
be292a9 bump release date
David Marin authored
107 v0.2.7, 2011-07-12 -- Hooray for interns!
7ee6a3c Ready for code review!
David Marin authored
108 * All runner options can be set from the command line (Issue #121)
109 * Including for mrjob.tools.emr.create_job_flow (Issue #142)
110 * New EMR options:
111 * availability_zone (Issue #72)
112 * bootstrap_actions (Issue #69)
113 * enable_emr_debugging (Issue #133)
114 * Read counters from EMR log files (Issue #134)
115 * Clean old files out of S3 with mrjob.tools.emr.s3_tmpwatch (Issue #9)
8035a41 updated CHANGES.txt and copyright notices
David Marin authored
116 * EMR parses and reports job failure due to steps timing out (Issue #15)
7ee6a3c Ready for code review!
David Marin authored
117 * EMR boostrap files are no longer made public on S3 (Issue #70)
118 * mrjob.tools.emr.terminate_idle_job_flows handles custom hadoop streaming
119 jars correctly (Issue #116)
120 * LocalMRJobRunner separates out counters by step (Issue #28)
121 * bootstrap_python_packages works regardless of tarball name (Issue #49)
122 * mrjob always creates temp buckets in the correct AWS region (Issue #64)
123 * Catch abuse of __main__ in jobs (Issue #78)
3ee6c4e added mr_travelling_salesman to CHANGES.txt
David Marin authored
124 * Added mr_travelling_salesman example
64a1cc2 updated CHANGES.txt to start tracking changes in v0.2.7
David Marin authored
125
f7d9061 prepare to release v0.2.6
David Marin authored
126 v0.2.6, 2011-05-24 -- Hadoop 0.20 in EMR, inline runner, and more
f4250b5 Merge branch 'master' into development
David Marin authored
127 * Set Hadoop to run on EMR with --hadoop-version (Issue #71).
c3c3975 finally, it's possible to use Hadoop 0.20 in EMR. (Issue #71)
David Marin authored
128 * Default is still 0.18, but will change to 0.20 in mrjob v0.3.0.
0947a1c updated CHANGES.txt
David Marin authored
129 * New inline runner, for testing locally with a debugger
58ca31a minor docstring change, added wqardaji to __credits__, updated CHANGES.t...
David Marin authored
130 * New --strict-protocols option, to catch unencodable data (Issue #76)
0947a1c updated CHANGES.txt
David Marin authored
131 * Added steps_python_bin option (for use with virtualenv)
34e70aa updated CHANGES.txt
David Marin authored
132 * mrjob no longer chokes when asked to run on an EMR job flow running
133 Hadoop 0.20 (Issue #110)
4feed57 mrjob no longer chokes on job flows with no LogUri (Issue #112)
David Marin authored
134 * mrjob no longer chokes on job flows with no LogUri (Issue #112)
0947a1c updated CHANGES.txt
David Marin authored
135
d05be28 v0.2.5 is ready to go!
David Marin authored
136 v0.2.5, 2011-04-29 -- Hadoop input and output formats
ad9aa85 updated release docs
David Marin authored
137 * Added hadoop_input/output_format options
138 * You can now specify a custom Hadoop streaming jar (hadoop_streaming_jar)
139 * extra args to hadoop now come before -mapper/-reducer on EMR, so
140 that e.g. -libjar will work (worked in hadoop mode since v0.2.2)
5507cdc Oops, S3 has URIs, not URLs
David Marin authored
141 * hadoop mode now supports s3n:// URIs (Issue #53)
ad9aa85 updated release docs
David Marin authored
142
3ac8ed0 add 505s to CHANGES.txt, change release date to today
David Marin authored
143 v0.2.4, 2011-03-09 -- fix bootstrapping mrjob
4868471 bump version number
David Marin authored
144 * Fix bootstrapping of mrjob in hadoop and local mode (Issue #89)
145 * SSH tunnels try to use the same port for the same job flow (Issue #67)
146 * Added mr_postfix_bounce and mr_pegasos_svm to examples.
3ac8ed0 add 505s to CHANGES.txt, change release date to today
David Marin authored
147 * Retry on spurious 505s from EMR API
4868471 bump version number
David Marin authored
148
3861740 bump version number
David Marin authored
149 v0.2.3, 2011-02-24 -- boto compatibility
150 * Fix incompatibility with boto 2.0b4 (Issue #91)
151
8adb2a7 Added --hadoop-arg fix to CHANGES.txt
David Marin authored
152 v0.2.2, 2011-02-15 -- GET/POST EMR issue
0030d3c oops, forgot to add GET/POST issue to CHANGES.txt
David Marin authored
153 * Use POST requests for most EMR queries (EMR was choking on large GETs)
70f9011 updated CHANGES.txt
David Marin authored
154 * find_probable_cause_of_failure() ignores transient errors (Issue #31)
8adb2a7 Added --hadoop-arg fix to CHANGES.txt
David Marin authored
155 * --hadoop-arg now actually works (Issue #79)
0030d3c oops, forgot to add GET/POST issue to CHANGES.txt
David Marin authored
156 * on Hadoop, extra args are added first, so you can set e.g. -libjar
ebe173c updated CHANGES.txt
David Marin authored
157 * S3 buckets may now have . in their names
158 * MRJob scripts now respect --quiet (Issue #84)
159 * added --no-output option for MRJob scripts (Issue #81)
160 * added --python-bin option (Issue #54)
8adb2a7 Added --hadoop-arg fix to CHANGES.txt
David Marin authored
161
3136331 Oops, wrong date in CHANGES.txt!
David Marin authored
162 v0.2.1, 2010-11-17 -- laststatechangereason bugfix
a8672db bump version number
David Marin authored
163 * Don't assume EMR sets laststatechangereason
164
ffef2bb updated CHANGES.txt
David Marin authored
165 v0.2.0, 2010-11-15 -- Many bugfixes, Windows support
166 * New Features/Changes:
148d0a7 Oops, weird word wrap
David Marin authored
167 * EMRJobRunner now prints % of mappers and reducers completed when you
168 enable the SSH tunnel.
ffef2bb updated CHANGES.txt
David Marin authored
169 * Added mr_page_rank example
170 * Added mrjob.tools.emr.audit_usage script (Issue #21)
171 * You can specify alternate job owners with the "owner" option. Useful for
172 auditing usage. (Issue #59)
173 * The job_name_prefix option has been renamed to label (the old name still
174 works but is deprecated)
9dec991 reword CHANGES.txt
David Marin authored
175 * bootstrap_cmds and bootstrap_scripts no longer automatically invoke sudo
ffef2bb updated CHANGES.txt
David Marin authored
176 * Bugs Fixed/Cleanup:
177 * bootstrap files no longer get uploaded to S3 twice (Issue #8)
178 * When using add_file_option(), show_steps() can now see the local version
179 of the file (Issue #45)
180 * Now works on Windows (Issue #46)
181 * No longer requires external jar, tar, or zip binaries (Issue #47)
182 * mrjob-* scratch bucket is only created as needed (Issue #50)
183 * Can now specify us-east-1 region explicitly (Issue #58)
184 * mrjob.tools.emr.terminate_idle_job_flows leaves Hive jobs alone (Issue #60)
185
f24991f bump version to v0.1.0
David Marin authored
186 v0.1.0, 2010-10-28 -- Same code, better version. It's official!
187
7a053fc administrivia for v0.1.0-pre3
David Marin authored
188 v0.1.0-pre3, 2010-10-27 -- Pre-release to run Yelp code against
189 * Added debian packaging
bf73c21 updated CHANGES.txt. Let's try this pre3 thing again. :)
David Marin authored
190 * mrjob bootstrapping can now deal with symlinks in site-packages/mrjob
7a053fc administrivia for v0.1.0-pre3
David Marin authored
191 * MRJobRunner.stream_output() can now be called multiple times
192
193 v0.1.0-pre2, 2010-10-25 -- Second pre-release after testing
a7e3100 bump version number to v0.1.0-pre2
David Marin authored
194 * Fixed small bugs that broke Python 2.5.1 and Python 2.7
195 * Fixed reading mrjob.conf without yaml installed
196 * Fix tests to work with modern simplejson and pipes.quote()
197 * Auto-create temp bucket on S3 if we don't have one (Issue #16)
198 * Auto-infer AWS region from bucket (Issue #7)
199 * --steps now passes in all extra args (e.g. --protocol) (Issue #4)
200 * Better docs
201
3af64ca Administrivia for initial pre-release.
David Marin authored
202 v0.1.0-pre1, 2010-10-21 -- Initial pre-release. YMMV!
Something went wrong with that request. Please try again.