Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

codemod: initial codemod release checkin.

  • Loading branch information...
commit 8da5577dede80c6ad40821c10ba5d9cad1efa4cf 0 parents
DaveFet authored December 17, 2008

Showing 2 changed files with 972 additions and 0 deletions. Show diff stats Hide diff stats

  1. 177  LICENSE
  2. 795  src/codemod.py
177  LICENSE
... ...
@@ -0,0 +1,177 @@
  1
+
  2
+                                 Apache License
  3
+                           Version 2.0, January 2004
  4
+                        http://www.apache.org/licenses/
  5
+
  6
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  7
+
  8
+   1. Definitions.
  9
+
  10
+      "License" shall mean the terms and conditions for use, reproduction,
  11
+      and distribution as defined by Sections 1 through 9 of this document.
  12
+
  13
+      "Licensor" shall mean the copyright owner or entity authorized by
  14
+      the copyright owner that is granting the License.
  15
+
  16
+      "Legal Entity" shall mean the union of the acting entity and all
  17
+      other entities that control, are controlled by, or are under common
  18
+      control with that entity. For the purposes of this definition,
  19
+      "control" means (i) the power, direct or indirect, to cause the
  20
+      direction or management of such entity, whether by contract or
  21
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
  22
+      outstanding shares, or (iii) beneficial ownership of such entity.
  23
+
  24
+      "You" (or "Your") shall mean an individual or Legal Entity
  25
+      exercising permissions granted by this License.
  26
+
  27
+      "Source" form shall mean the preferred form for making modifications,
  28
+      including but not limited to software source code, documentation
  29
+      source, and configuration files.
  30
+
  31
+      "Object" form shall mean any form resulting from mechanical
  32
+      transformation or translation of a Source form, including but
  33
+      not limited to compiled object code, generated documentation,
  34
+      and conversions to other media types.
  35
+
  36
+      "Work" shall mean the work of authorship, whether in Source or
  37
+      Object form, made available under the License, as indicated by a
  38
+      copyright notice that is included in or attached to the work
  39
+      (an example is provided in the Appendix below).
  40
+
  41
+      "Derivative Works" shall mean any work, whether in Source or Object
  42
+      form, that is based on (or derived from) the Work and for which the
  43
+      editorial revisions, annotations, elaborations, or other modifications
  44
+      represent, as a whole, an original work of authorship. For the purposes
  45
+      of this License, Derivative Works shall not include works that remain
  46
+      separable from, or merely link (or bind by name) to the interfaces of,
  47
+      the Work and Derivative Works thereof.
  48
+
  49
+      "Contribution" shall mean any work of authorship, including
  50
+      the original version of the Work and any modifications or additions
  51
+      to that Work or Derivative Works thereof, that is intentionally
  52
+      submitted to Licensor for inclusion in the Work by the copyright owner
  53
+      or by an individual or Legal Entity authorized to submit on behalf of
  54
+      the copyright owner. For the purposes of this definition, "submitted"
  55
+      means any form of electronic, verbal, or written communication sent
  56
+      to the Licensor or its representatives, including but not limited to
  57
+      communication on electronic mailing lists, source code control systems,
  58
+      and issue tracking systems that are managed by, or on behalf of, the
  59
+      Licensor for the purpose of discussing and improving the Work, but
  60
+      excluding communication that is conspicuously marked or otherwise
  61
+      designated in writing by the copyright owner as "Not a Contribution."
  62
+
  63
+      "Contributor" shall mean Licensor and any individual or Legal Entity
  64
+      on behalf of whom a Contribution has been received by Licensor and
  65
+      subsequently incorporated within the Work.
  66
+
  67
+   2. Grant of Copyright License. Subject to the terms and conditions of
  68
+      this License, each Contributor hereby grants to You a perpetual,
  69
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
  70
+      copyright license to reproduce, prepare Derivative Works of,
  71
+      publicly display, publicly perform, sublicense, and distribute the
  72
+      Work and such Derivative Works in Source or Object form.
  73
+
  74
+   3. Grant of Patent License. Subject to the terms and conditions of
  75
+      this License, each Contributor hereby grants to You a perpetual,
  76
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
  77
+      (except as stated in this section) patent license to make, have made,
  78
+      use, offer to sell, sell, import, and otherwise transfer the Work,
  79
+      where such license applies only to those patent claims licensable
  80
+      by such Contributor that are necessarily infringed by their
  81
+      Contribution(s) alone or by combination of their Contribution(s)
  82
+      with the Work to which such Contribution(s) was submitted. If You
  83
+      institute patent litigation against any entity (including a
  84
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
  85
+      or a Contribution incorporated within the Work constitutes direct
  86
+      or contributory patent infringement, then any patent licenses
  87
+      granted to You under this License for that Work shall terminate
  88
+      as of the date such litigation is filed.
  89
+
  90
+   4. Redistribution. You may reproduce and distribute copies of the
  91
+      Work or Derivative Works thereof in any medium, with or without
  92
+      modifications, and in Source or Object form, provided that You
  93
+      meet the following conditions:
  94
+
  95
+      (a) You must give any other recipients of the Work or
  96
+          Derivative Works a copy of this License; and
  97
+
  98
+      (b) You must cause any modified files to carry prominent notices
  99
+          stating that You changed the files; and
  100
+
  101
+      (c) You must retain, in the Source form of any Derivative Works
  102
+          that You distribute, all copyright, patent, trademark, and
  103
+          attribution notices from the Source form of the Work,
  104
+          excluding those notices that do not pertain to any part of
  105
+          the Derivative Works; and
  106
+
  107
+      (d) If the Work includes a "NOTICE" text file as part of its
  108
+          distribution, then any Derivative Works that You distribute must
  109
+          include a readable copy of the attribution notices contained
  110
+          within such NOTICE file, excluding those notices that do not
  111
+          pertain to any part of the Derivative Works, in at least one
  112
+          of the following places: within a NOTICE text file distributed
  113
+          as part of the Derivative Works; within the Source form or
  114
+          documentation, if provided along with the Derivative Works; or,
  115
+          within a display generated by the Derivative Works, if and
  116
+          wherever such third-party notices normally appear. The contents
  117
+          of the NOTICE file are for informational purposes only and
  118
+          do not modify the License. You may add Your own attribution
  119
+          notices within Derivative Works that You distribute, alongside
  120
+          or as an addendum to the NOTICE text from the Work, provided
  121
+          that such additional attribution notices cannot be construed
  122
+          as modifying the License.
  123
+
  124
+      You may add Your own copyright statement to Your modifications and
  125
+      may provide additional or different license terms and conditions
  126
+      for use, reproduction, or distribution of Your modifications, or
  127
+      for any such Derivative Works as a whole, provided Your use,
  128
+      reproduction, and distribution of the Work otherwise complies with
  129
+      the conditions stated in this License.
  130
+
  131
+   5. Submission of Contributions. Unless You explicitly state otherwise,
  132
+      any Contribution intentionally submitted for inclusion in the Work
  133
+      by You to the Licensor shall be under the terms and conditions of
  134
+      this License, without any additional terms or conditions.
  135
+      Notwithstanding the above, nothing herein shall supersede or modify
  136
+      the terms of any separate license agreement you may have executed
  137
+      with Licensor regarding such Contributions.
  138
+
  139
+   6. Trademarks. This License does not grant permission to use the trade
  140
+      names, trademarks, service marks, or product names of the Licensor,
  141
+      except as required for reasonable and customary use in describing the
  142
+      origin of the Work and reproducing the content of the NOTICE file.
  143
+
  144
+   7. Disclaimer of Warranty. Unless required by applicable law or
  145
+      agreed to in writing, Licensor provides the Work (and each
  146
+      Contributor provides its Contributions) on an "AS IS" BASIS,
  147
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
  148
+      implied, including, without limitation, any warranties or conditions
  149
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
  150
+      PARTICULAR PURPOSE. You are solely responsible for determining the
  151
+      appropriateness of using or redistributing the Work and assume any
  152
+      risks associated with Your exercise of permissions under this License.
  153
+
  154
+   8. Limitation of Liability. In no event and under no legal theory,
  155
+      whether in tort (including negligence), contract, or otherwise,
  156
+      unless required by applicable law (such as deliberate and grossly
  157
+      negligent acts) or agreed to in writing, shall any Contributor be
  158
+      liable to You for damages, including any direct, indirect, special,
  159
+      incidental, or consequential damages of any character arising as a
  160
+      result of this License or out of the use or inability to use the
  161
+      Work (including but not limited to damages for loss of goodwill,
  162
+      work stoppage, computer failure or malfunction, or any and all
  163
+      other commercial damages or losses), even if such Contributor
  164
+      has been advised of the possibility of such damages.
  165
+
  166
+   9. Accepting Warranty or Additional Liability. While redistributing
  167
+      the Work or Derivative Works thereof, You may choose to offer,
  168
+      and charge a fee for, acceptance of support, warranty, indemnity,
  169
+      or other liability obligations and/or rights consistent with this
  170
+      License. However, in accepting such obligations, You may act only
  171
+      on Your own behalf and on Your sole responsibility, not on behalf
  172
+      of any other Contributor, and only if You agree to indemnify,
  173
+      defend, and hold each Contributor harmless for any liability
  174
+      incurred by, or claims asserted against, such Contributor by reason
  175
+      of your accepting any such warranty or additional liability.
  176
+
  177
+   END OF TERMS AND CONDITIONS
795  src/codemod.py
... ...
@@ -0,0 +1,795 @@
  1
+#!/usr/bin/env python
  2
+
  3
+# Copyright (c) 2007-2008 Facebook
  4
+#
  5
+# Licensed under the Apache License, Version 2.0 (the "License");
  6
+# you may not use this file except in compliance with the License.
  7
+# You may obtain a copy of the License at
  8
+#
  9
+#     http://www.apache.org/licenses/LICENSE-2.0
  10
+#
  11
+# Unless required by applicable law or agreed to in writing, software
  12
+# distributed under the License is distributed on an "AS IS" BASIS,
  13
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  14
+# See the License for the specific language governing permissions and
  15
+# limitations under the License.
  16
+#
  17
+# See accompanying file LICENSE.
  18
+#
  19
+# @author Justin Rosenstein
  20
+
  21
+r"""
  22
+codemod.py is a tool/library to assist you with large-scale codebase refactors
  23
+that can be partially automated but still require human oversight and
  24
+occassional intervention.
  25
+
  26
+Example: Let's say you're deprecating your use of the <font> tag.  From the
  27
+command line, you might make progress by running:
  28
+
  29
+  codemod.py -m -d /home/jrosenstein/www --extensions php,html \
  30
+             '<font *color="?(.*?)"?>(.*?)</font>' \
  31
+             '<span style="color: \1;">\2</span>'
  32
+
  33
+For each match of the regex, you'll be shown a colored diff, and asked if you
  34
+want to accept the change (the replacement of the <font> tag with a <span>
  35
+tag), reject it, or edit the line in question in your $EDITOR of choice.
  36
+
  37
+Usage: last two arguments are a regular expression to match and a substitution
  38
+       string, respectively.  Or you can omit the substitution string, and just
  39
+       be prompted on each match for whether you want to edit in your editor.
  40
+
  41
+Options (all optional) include:
  42
+
  43
+  -m
  44
+    Have regex work over multiple lines (e.g. have dot match newlines).  By
  45
+    default, codemod applies the regex one line at a time.
  46
+  -d
  47
+    The path whose ancestor files are to be explored.  Defaults to current dir.
  48
+  --start
  49
+    A path:line_number-formatted position somewhere in the hierarchy from which
  50
+    to being exploring, or a percentage (e.g. "--start 25%") of the way through
  51
+    to start.  Useful if you're divvying up the substitution task across
  52
+    multiple people.
  53
+  --end
  54
+    A path:line_number-formatted position somewhere in the hierarchy just
  55
+    *before* which we should stop exploring, or a percentage of the way
  56
+    through, just before which to end.
  57
+  --extensions
  58
+    A comma-delimited list of file extensions to process.
  59
+  --editor
  60
+    Specify an editor, e.g. "vim" or "emacs".  If omitted, defaults to $EDITOR
  61
+    environment variable.
  62
+  --count
  63
+    Don't run normally.  Instead, just print out number of times places in the
  64
+    codebase where the 'query' matches.
  65
+  --test
  66
+    Don't run normally.  Instead, just run the unit tests embedded in the
  67
+    codemod library.
  68
+
  69
+You can also use codemod for transformations that are much more sophisticated
  70
+than regular expression substitution.  Rather than using the command line, you
  71
+write Python code that looks like:
  72
+
  73
+  import codemod
  74
+  codemod.Query(...).run_interactive()
  75
+
  76
+See the documentation for the Query class for details.
  77
+
  78
+@author Justin Rosenstein
  79
+"""
  80
+
  81
+
  82
+import sys, os
  83
+
  84
+def path_filter(extensions=None, exclude_paths=[]):
  85
+  """
  86
+  Returns a function (useful as the path_filter field of a Query instance)
  87
+  that returns True iff the path it is given has an extension one of the
  88
+  file extensions specified in `extensions`, an array of strings.
  89
+
  90
+  >>> map(path_filter(extensions=['js', 'php']), ['./profile.php', './q.jjs'])
  91
+  [True, False]
  92
+  >>> map(path_filter(exclude_paths=['html']), ['./html/x.php', './lib/y.js'])
  93
+  [False, True]
  94
+  """
  95
+  def the_filter(path):
  96
+    if extensions:
  97
+      if not any(path.endswith('.' + extension) for extension in extensions):
  98
+        return False
  99
+    for excluded in exclude_paths:
  100
+      if path.startswith(excluded) or path.startswith('./' + excluded):
  101
+        return False
  102
+    return True
  103
+  return the_filter
  104
+
  105
+_default_path_filter = path_filter(extensions=['php', 'phpt', 'js', 'css'])
  106
+
  107
+def run_interactive(query, editor=None, just_count=False):
  108
+  """
  109
+  Asks the user about each patch suggested by the result of the query.
  110
+
  111
+  @param query        An instance of the Query class.
  112
+  @param editor       Name of editor to use for manual intervention, e.g. 'vim'
  113
+                      or 'emacs'.  If omitted/None, defaults to $EDITOR
  114
+                      environment variable.
  115
+  @param just_count   If true: don't run normally.  Just print out number of
  116
+                      places in the codebase where the query matches.
  117
+  """
  118
+
  119
+  # Load start from bookmark, if appropriate.
  120
+  bookmark = _load_bookmark()
  121
+  if bookmark:
  122
+    print 'Resume where you left off, at %s (y/n)? ' % str(bookmark),
  123
+    if (_prompt(default='y') == 'y'):
  124
+      query.start_position = bookmark
  125
+
  126
+  # Okay, enough of this foolishness of computing start and end.
  127
+  # Let's ask the user about some one line diffs!
  128
+  print 'Searching for first instance...'
  129
+  suggestions = query.generate_patches()
  130
+
  131
+  if just_count:
  132
+    for count, _ in enumerate(suggestions):
  133
+      terminal_move_to_beginning_of_line()
  134
+      print count,
  135
+      sys.stdout.flush()  # since print statement ends in comma
  136
+    print
  137
+    return
  138
+
  139
+  for patch in suggestions:
  140
+    _save_bookmark(patch.start_position)
  141
+    _ask_about_patch(patch, editor)
  142
+    print 'Searching...'
  143
+  _delete_bookmark()
  144
+
  145
+def line_transformation_suggestor(line_transformation, line_filter=None):
  146
+  """
  147
+  Returns a suggestor (a function that takes a list of lines and yields
  148
+  patches) where suggestions are the result of line-by-line transformations.
  149
+
  150
+  @param line_transformation  Function that, given a line, returns another line
  151
+                              with which to replace the given one.  If the
  152
+                              output line is different from the input line, the
  153
+                              user will be prompted about whether to make the
  154
+                              change.  If the output is None, this means "I
  155
+                              don't have a suggestion, but the user should
  156
+                              still be asked if zhe wants to edit the line."
  157
+  @param line_filter          Given a line, returns True or False.  If False,
  158
+                              a line is ignored (as if line_transformation
  159
+                              returned the line itself for that line).
  160
+  """
  161
+  def suggestor(lines):
  162
+    for line_number, line in enumerate(lines):
  163
+      if line_filter and not line_filter(line):
  164
+        continue
  165
+      candidate = line_transformation(line)
  166
+      if candidate is None:
  167
+        yield Patch(line_number)
  168
+      else:
  169
+        yield Patch(line_number, new_lines=[candidate])
  170
+  return suggestor
  171
+
  172
+def regex_suggestor(regex, substitution=None, line_filter=None):
  173
+  if isinstance(regex, str):
  174
+    import re
  175
+    regex = re.compile(regex)
  176
+
  177
+  if substitution is None:
  178
+    line_transformation = lambda line: None if regex.search(line) else line
  179
+  else:
  180
+    line_transformation = lambda line: regex.sub(substitution, line)
  181
+  return line_transformation_suggestor(line_transformation, line_filter)
  182
+
  183
+def multiline_regex_suggestor(regex, substitution=None):
  184
+  """
  185
+  Return a suggestor function which, given a list of lines, generates patches
  186
+  to substitute matches of the given regex with (if provided) the given
  187
+  substitution.
  188
+
  189
+  @param regex         Either a regex object or a string describing a regex.
  190
+  @param substitution  Either None (meaning that we should flag the matches
  191
+                       without suggesting an alternative), or a string (using
  192
+                       \1 notation to backreference match groups) or a
  193
+                       function (that takes a match object as input).
  194
+  """
  195
+  import re
  196
+  if isinstance(regex, str):
  197
+    regex = re.compile(regex, re.DOTALL)
  198
+
  199
+  if isinstance(substitution, str):
  200
+    substitution_func = lambda match: match.expand(substitution)
  201
+  else:
  202
+    substitution_func = substitution
  203
+
  204
+  def suggestor(lines):
  205
+    pos = 0
  206
+    while True:
  207
+      match = regex.search(''.join(lines), pos)
  208
+      if not match:
  209
+        break
  210
+      start_row, start_col = _index_to_row_col(lines, match.start())
  211
+      end_row,   end_col   = _index_to_row_col(lines, match.end()-1)
  212
+
  213
+      if substitution is None:
  214
+        new_lines = None
  215
+      else:
  216
+        # TODO: ugh, this is hacky.  Clearly I need to rewrite this to use
  217
+        #       character-level patches, rather than line-level patches.
  218
+        new_lines = substitution_func(match)
  219
+        if new_lines is not None:
  220
+          new_lines = ''.join((
  221
+            lines[start_row][:start_col],
  222
+            new_lines,
  223
+            lines[end_row][end_col+1:]
  224
+          ))
  225
+
  226
+      yield Patch(
  227
+        start_line_number = start_row,
  228
+        end_line_number   = end_row + 1,
  229
+        new_lines         = new_lines
  230
+      )
  231
+      pos = match.start() + 1
  232
+
  233
+  return suggestor
  234
+
  235
+def _index_to_row_col(lines, index):
  236
+  r"""
  237
+  >>> lines = ['hello\n', 'world\n']
  238
+  >>> _index_to_row_col(lines, 0)
  239
+  (0, 0)
  240
+  >>> _index_to_row_col(lines, 7)
  241
+  (1, 1)
  242
+  """
  243
+  if index < 0:
  244
+    raise IndexError('negative index')
  245
+  current_index = 0
  246
+  for line_number, line in enumerate(lines):
  247
+    line_length = len(line)
  248
+    if current_index + line_length > index:
  249
+      return line_number, index - current_index
  250
+    current_index += line_length
  251
+  raise IndexError('index %d out of range' % index)
  252
+
  253
+
  254
+class Query:
  255
+  """
  256
+  Represents a suggestor, along with a set of constraints on which files
  257
+  should be fed to that suggestor.
  258
+
  259
+  >>> Query(lambda x: None, start='profile.php:20').start_position
  260
+  Position('profile.php', 20)
  261
+  """
  262
+
  263
+  def __init__(self,
  264
+               suggestor,
  265
+               start=None,
  266
+               end=None,
  267
+               root_directory='.',
  268
+               path_filter=_default_path_filter):
  269
+    """
  270
+    @param suggestor            A function that takes a list of lines and
  271
+                                generates instances of Patch to suggest.
  272
+                                (Patches should not specify paths.)
  273
+    @param start                One of:
  274
+                                - an instance of Position (indicating the place
  275
+                                  in the file hierarchy at which to resume),
  276
+                                - a path:line_number-formatted string
  277
+                                  representing a position,
  278
+                                - a string formatted like "25%" (indicating we
  279
+                                  should start 25% of the way through the
  280
+                                  process), or
  281
+                                - None (indicating that we should start at the
  282
+                                  beginning).
  283
+    @param end                  An indicator of the position just *before* which
  284
+                                to stop exploring, using one of the same formats
  285
+                                used for start (where None means 'traverse to
  286
+                                the end of the hierarchy).
  287
+    @param root_directory       The path whose ancestor files are to be explored.
  288
+    @param path_filter          Given a path, returns True or False.  If False,
  289
+                                the entire file is ignored.
  290
+    """
  291
+    self.suggestor          = suggestor
  292
+    self._start             = start
  293
+    self._end               = end
  294
+    self.root_directory     = root_directory
  295
+    self.path_filter        = path_filter
  296
+    self._all_patches_cache = None
  297
+
  298
+  def clone(self):
  299
+    import copy
  300
+    return copy.copy(self)
  301
+
  302
+  def _get_position(self, attr_name):
  303
+    attr_value = getattr(self, attr_name)
  304
+    if attr_value is None:
  305
+      return None
  306
+    if isinstance(attr_value, str) and attr_value.endswith('%'):
  307
+      attr_value = self.compute_percentile(int(attr_value[:-1]))
  308
+      setattr(self, attr_name, attr_value)
  309
+    return Position(attr_value)
  310
+
  311
+  def get_start_position(self):
  312
+    return self._get_position('_start')
  313
+  start_position = property(get_start_position)
  314
+
  315
+  def get_end_position(self):
  316
+    return self._get_position('_end')
  317
+  end_position = property(get_end_position)
  318
+
  319
+  def get_all_patches(self, dont_use_cache=False):
  320
+    """
  321
+    Computes a list of all patches matching this query, though ignoreing
  322
+    self.start_position and self.end_position.
  323
+
  324
+    @param dont_use_cache   If False, and get_all_patches has been called
  325
+                            before, compute the list computed last time.
  326
+    """
  327
+    if not dont_use_cache and self._all_patches_cache is not None:
  328
+      return self._all_patches_cache
  329
+
  330
+    print 'Computing full change list (since you specified a percentage)...',
  331
+    sys.stdout.flush()  # since print statement ends in comma
  332
+
  333
+    endless_query = self.clone()
  334
+    endless_query.start_position = endless_query.end_position = None
  335
+    self._all_patches_cache = list(endless_query.generate_patches())
  336
+    return self._all_patches_cache
  337
+
  338
+  def compute_percentile(self, percentage):
  339
+    """
  340
+    Returns a Position object that represents percentage%-far-of-the-way
  341
+    through the larger task, as specified by this query.
  342
+
  343
+    @param percentage    a number between 0 and 100.
  344
+    """
  345
+    all_patches = self.get_all_patches()
  346
+    return all_patches[int(len(all_patches) * percentage / 100)].start_position
  347
+
  348
+  def generate_patches(self):
  349
+    """
  350
+    Generates a list of patches for each file underneath self.root_directory
  351
+    that satisfy the given conditions given query conditions, where patches for
  352
+    each file are suggested by self.suggestor.
  353
+    """
  354
+    start_pos = self.start_position or Position(None, None)
  355
+    end_pos   = self.end_position   or Position(None, None)
  356
+
  357
+    path_list = Query._walk_directory(self.root_directory)
  358
+    path_list = Query._sublist(path_list, start_pos.path, end_pos.path)
  359
+    path_list = (path for path in path_list if
  360
+                 Query._path_looks_like_code(path) and self.path_filter(path))
  361
+
  362
+    for path in path_list:
  363
+
  364
+      lines = list(open(path))
  365
+      for patch in self.suggestor(lines):
  366
+        if path == start_pos.path:
  367
+          if patch.start_line_number < start_pos.line_number:
  368
+            continue  # suggestion is pre-start_pos
  369
+        if path == end_pos.path:
  370
+          if patch.end_line_number >= end_pos.line_number:
  371
+            break  # suggestion is post-end_pos
  372
+
  373
+        old_lines = lines[patch.start_line_number:patch.end_line_number]
  374
+        if patch.new_lines is None or patch.new_lines != old_lines:
  375
+          patch.path = path
  376
+          yield patch
  377
+          lines[:] = list(open(path))  # re-open file, in case contents changed
  378
+
  379
+  def run_interactive(self, **kargs):
  380
+    run_interactive(self, **kargs)
  381
+
  382
+  @staticmethod
  383
+  def _walk_directory(root_directory):
  384
+    """
  385
+    Generates the paths of all files that are ancestors of `root_directory`.
  386
+    """
  387
+
  388
+    paths = [os.path.join(root, name)
  389
+             for root, dirs, files in os.walk(root_directory)
  390
+             for name in files]
  391
+    paths.sort()
  392
+    return paths
  393
+
  394
+  @staticmethod
  395
+  def _sublist(list, starting_value, ending_value = None):
  396
+    """
  397
+    >>> list(Query._sublist((x*x for x in xrange(1, 100)), 16, 64))
  398
+    [16, 25, 36, 49, 64]
  399
+    """
  400
+    have_started = starting_value is None
  401
+
  402
+    for x in list:
  403
+      have_started = have_started or x == starting_value
  404
+      if have_started:
  405
+        yield x
  406
+
  407
+      if ending_value is not None and x == ending_value:
  408
+        break
  409
+
  410
+  @staticmethod
  411
+  def _path_looks_like_code(path):
  412
+    """
  413
+    >>> Query._path_looks_like_code('/home/jrosenstein/www/profile.php')
  414
+    True
  415
+    >>> Query._path_looks_like_code('./tags')
  416
+    False
  417
+    >>> Query._path_looks_like_code('/home/jrosenstein/www/profile.php~')
  418
+    False
  419
+    >>> Query._path_looks_like_code('/home/jrosenstein/www/.git/HEAD')
  420
+    False
  421
+    """
  422
+    return ('/.' not in path and path[-1] != '~'
  423
+            and not path.endswith('tags')
  424
+            and not path.endswith('TAGS'))
  425
+
  426
+
  427
+class Position:
  428
+  """
  429
+  >>> p1, p2 = Position('./hi.php', 20), Position('./hi.php:20')
  430
+  >>> p1.path == p2.path and p1.line_number == p2.line_number
  431
+  True
  432
+  >>> p1
  433
+  Position('./hi.php', 20)
  434
+  >>> print p1
  435
+  ./hi.php:20
  436
+  >>> Position(p1)
  437
+  Position('./hi.php', 20)
  438
+  """
  439
+
  440
+  def __init__(self, *path_and_line_number):
  441
+    """
  442
+    You can use the two parameter version, and pass a path and line number, or
  443
+    you can use the one parameter version, and pass a $path:$line_number string,
  444
+    or another instance of Position to copy.
  445
+    """
  446
+    if len(path_and_line_number) == 2:
  447
+      self.path, self.line_number = path_and_line_number
  448
+    elif len(path_and_line_number) == 1:
  449
+      arg = path_and_line_number[0]
  450
+      if isinstance(arg, Position):
  451
+        self.path, self.line_number = arg.path, arg.line_number
  452
+      else:
  453
+        try:
  454
+          self.path, line_number_s = arg.split(':')
  455
+          self.line_number = int(line_number_s)
  456
+        except ValueError:
  457
+          raise ValueError('inappropriately formatted Position string: %s'
  458
+                           % path_and_line_number[0])
  459
+    else:
  460
+      raise TypeError('Position takes 1 or 2 arguments')
  461
+
  462
+  def __repr__(self):
  463
+    return 'Position(%s, %d)' % (repr(self.path), self.line_number)
  464
+
  465
+  def __str__(self):
  466
+    return '%s:%d' % (self.path, self.line_number)
  467
+
  468
+
  469
+class Patch:
  470
+  """
  471
+  Represents a range of a file and (optionally) a list of lines with which to
  472
+  replace that range.
  473
+
  474
+  >>> p = Patch(2, 4, ['X', 'Y', 'Z'], 'x.php')
  475
+  >>> print p.render_range()
  476
+  x.php:2-3
  477
+  >>> p.start_position
  478
+  Position('x.php', 2)
  479
+  >>> l = ['a', 'b', 'c', 'd', 'e', 'f']
  480
+  >>> p.apply_to(l)
  481
+  >>> l
  482
+  ['a', 'b', 'X', 'Y', 'Z', 'e', 'f']
  483
+  """
  484
+  def __init__(self, start_line_number, end_line_number=None, new_lines=None,
  485
+               path=None):
  486
+    """
  487
+    Constructs a Patch object.
  488
+
  489
+    @param end_line_number  The line number just *after* the end of the range.
  490
+                            Defaults to start_line_number + 1, i.e. a one-line
  491
+                            diff.
  492
+    @param new_lines        The set of lines with which to replace the range
  493
+                            specified, or a newline-delimited string.  Omitting
  494
+                            this means that this "patch" doesn't actually
  495
+                            suggest a change.
  496
+    @param path             Path is optional only so that suggestors that have
  497
+                            been passed a list of lines don't have to set the
  498
+                            path explicitly.  (It'll get set by the suggestor's
  499
+                            caller.)
  500
+    """
  501
+    if end_line_number is None:
  502
+      end_line_number = start_line_number + 1
  503
+    if isinstance(new_lines, str):
  504
+      new_lines = new_lines.splitlines(True)
  505
+    for k, v in locals().iteritems():
  506
+      setattr(self, k, v)
  507
+  def __repr__(self):
  508
+    return 'Patch()' % ', '.join(map(repr, [
  509
+        self.path, self.start_line_number, self.end_line_number, self.new_lines
  510
+      ]))
  511
+  def apply_to(self, lines):
  512
+    if self.new_lines is None:
  513
+      raise ValueError('Can\'t apply patch without suggested new lines.')
  514
+    lines[self.start_line_number:self.end_line_number] = self.new_lines
  515
+  def render_range(self):
  516
+    path = self.path or '<unknown>'
  517
+    if self.start_line_number == self.end_line_number - 1:
  518
+      return '%s:%d' % (path, self.start_line_number)
  519
+    else:
  520
+      return '%s:%d-%d' % (path,
  521
+          self.start_line_number, self.end_line_number - 1)
  522
+  def get_start_position(self):
  523
+    return Position(self.path, self.start_line_number)
  524
+  start_position = property(get_start_position)
  525
+
  526
+
  527
+def print_patch(patch, lines_to_print, file_lines=None):
  528
+  from math import floor, ceil
  529
+
  530
+  if file_lines is None:
  531
+    file_lines = list(open(patch.path))
  532
+
  533
+  size_of_old               = patch.end_line_number - patch.start_line_number
  534
+  size_of_new               = len(patch.new_lines) if patch.new_lines else 0
  535
+  size_of_diff              = size_of_old + size_of_new
  536
+  size_of_context           = max(0, lines_to_print - size_of_diff)
  537
+  size_of_up_context        = int(size_of_context / 2)
  538
+  size_of_down_context      = int(ceil(size_of_context / 2))
  539
+  start_context_line_number = patch.start_line_number - size_of_up_context
  540
+  end_context_line_number   = patch.end_line_number + size_of_down_context
  541
+
  542
+  def print_file_line(line_number):
  543
+    print ('  %s' % file_lines[i]) if (0 <= i < len(file_lines)) else '~\n',
  544
+
  545
+  for i in xrange(start_context_line_number, patch.start_line_number):
  546
+    print_file_line(i)
  547
+  for i in xrange(patch.start_line_number, patch.end_line_number):
  548
+    if patch.new_lines is not None:
  549
+      terminal_print('- %s' % file_lines[i], color='RED')
  550
+    else:
  551
+      terminal_print('* %s' % file_lines[i], color='YELLOW')
  552
+  if patch.new_lines is not None:
  553
+    for line in patch.new_lines:
  554
+      terminal_print('+ %s' % line, color='GREEN')
  555
+  for i in xrange(patch.end_line_number, end_context_line_number):
  556
+    print_file_line(i)
  557
+
  558
+
  559
+def _ask_about_patch(patch, editor):
  560
+  terminal_clear()
  561
+  terminal_print('%s\n' % patch.render_range(), color='WHITE')
  562
+  print
  563
+
  564
+  lines = list(open(patch.path))
  565
+  print_patch(patch, terminal_get_size()[0] - 20, lines)
  566
+
  567
+  print
  568
+
  569
+  if patch.new_lines is not None:
  570
+    print 'Accept change (y = yes [default], n = no, e = edit, E = yes+edit)? ',
  571
+    p = _prompt('yneE', default='y')
  572
+  else:
  573
+    print '(e = edit [default], n = skip line)? ',
  574
+    p = _prompt('en', default='e')
  575
+
  576
+  if p in 'yE':
  577
+    patch.apply_to(lines)
  578
+    _save(patch.path, lines)
  579
+  if p in 'eE':
  580
+    run_editor(patch.start_position, editor)
  581
+
  582
+
  583
+def _prompt(letters='yn', default=None):
  584
+  """
  585
+  Wait for the user to type a character (and hit Enter).  If the user enters
  586
+  one of the characters in `letters`, return that character.  If the user
  587
+  hits Enter without entering a character, and `default` is specified, returns
  588
+  `default`.  Otherwise, asks the user to enter a character again.
  589
+  """
  590
+  while True:
  591
+    try:
  592
+      input = sys.stdin.readline().strip()
  593
+    except KeyboardInterrupt:
  594
+      sys.exit(0)
  595
+    if input and input in letters:
  596
+      return input
  597
+    if default is not None and input == '':
  598
+      return default
  599
+    print 'Come again?'
  600
+
  601
+def _save(path, lines):
  602
+  file_w = open(path, 'w')
  603
+  for line in lines:
  604
+    file_w.write(line)
  605
+  file_w.close()
  606
+
  607
+def run_editor(position, editor=None):
  608
+  editor = editor or os.environ.get('EDITOR') or 'vim'
  609
+  os.system('%s +%d %s' % (editor, position.line_number + 1, position.path))
  610
+
  611
+
  612
+#
  613
+# Bookmarking functions.  codemod saves a file called .codemod.bookmark to
  614
+# keep track of where you were the last time you exited in the middle of
  615
+# an interactive sesh.
  616
+#
  617
+
  618
+def _save_bookmark(position):
  619
+  file_w = open('.codemod.bookmark', 'w')
  620
+  file_w.write(str(position))
  621
+  file_w.close()
  622
+
  623
+def _load_bookmark():
  624
+  try:
  625
+    file = open('.codemod.bookmark')
  626
+  except IOError:
  627
+    return None
  628
+  contents = file.readline().strip()
  629
+  file.close()
  630
+  return Position(contents)
  631
+
  632
+def _delete_bookmark():
  633
+  try:
  634
+    os.remove('.codemod.bookmark')
  635
+  except OSError:
  636
+    pass  # file didn't exist
  637
+
  638
+
  639
+#
  640
+# Functions for working with the terminal.  Should probably be moved to a
  641
+# standalone library.
  642
+#
  643
+
  644
+def terminal_get_size(default_size = (25, 80)):
  645
+  """
  646
+  Return (number of rows, number of columns) for the terminal, if they can be determined,
  647
+  or `default_size` if they can't.
  648
+  """
  649
+
  650
+  def ioctl_GWINSZ(fd):                  #### TABULATION FUNCTIONS
  651
+    try:                                ### Discover terminal width
  652
+      import fcntl, termios, struct, os
  653
+      return struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234'))
  654
+    except:
  655
+      return None
  656
+
  657
+  # try open fds
  658
+  size = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
  659
+  if not size:
  660
+    # ...then ctty
  661
+    try:
  662
+      fd = os.open(os.ctermid(), os.O_RDONLY)
  663
+      size = ioctl_GWINSZ(fd)
  664
+      os.close(fd)
  665
+    except:
  666
+      pass
  667
+  if not size:
  668
+    # env vars or finally defaults
  669
+    try:
  670
+      size = (env['LINES'], env['COLUMNS'])
  671
+    except:
  672
+      return default_size
  673
+
  674
+  return map(int, size)
  675
+
  676
+def terminal_clear():
  677
+  """
  678
+  Like calling the `clear` UNIX command.  If that fails, just prints a bunch
  679
+  of newlines :-P
  680
+  """
  681
+  if not _terminal_use_capability('clear'):
  682
+    print '\n' * 8
  683
+
  684
+def terminal_move_to_beginning_of_line():
  685
+  """
  686
+  Jumps the cursor back to the beginning of the current line of text.
  687
+  """
  688
+  if not _terminal_use_capability('cr'):
  689
+    print
  690
+
  691
+def _terminal_use_capability(capability_name):
  692
+  """
  693
+  If the terminal supports the given capability, output it.  Return whether
  694
+  it was output.
  695
+  """
  696
+  import curses, sys
  697
+  curses.setupterm()
  698
+  capability = curses.tigetstr(capability_name)
  699
+  if capability:
  700
+    sys.stdout.write(capability)
  701
+  return bool(capability)
  702
+
  703
+def terminal_print(text, color):
  704
+  """Print text in the specified color, without a terminating newline."""
  705
+  _terminal_set_color(color)
  706
+  print text,
  707
+  _terminal_restore_color()
  708
+
  709
+def _terminal_set_color(color):
  710
+  import curses, sys
  711
+  def color_code(set_capability, possible_colors):
  712
+    try:
  713
+      color_index = possible_colors.split(' ').index(color)
  714
+    except ValueError:
  715
+      return None
  716
+    set_code = curses.tigetstr(set_capability)
  717
+    if not set_code:
  718
+      return None
  719
+    return curses.tparm(set_code, color_index)
  720
+  code = (color_code('setaf', 'BLACK RED GREEN YELLOW BLUE MAGENTA CYAN WHITE')
  721
+       or color_code('setf', 'BLACK BLUE GREEN CYAN RED MAGENTA YELLOW WHITE'))
  722
+  if code:
  723
+    sys.stdout.write(code)
  724
+
  725
+def _terminal_restore_color():
  726
+  import curses, sys
  727
+  sys.stdout.write(curses.tigetstr('sgr0'))
  728
+
  729
+def print_through_less(text):
  730
+  """
  731
+  Prints `text` to standard output.  If `text` wouldn't fit on one screen (as
  732
+  measured by line count), make output scrollable a la `less`.
  733
+  """
  734
+  from tempfile import NamedTemporaryFile
  735
+  tempfile = NamedTemporaryFile()
  736
+  tempfile.write(text)
  737
+  tempfile.flush()
  738
+  os.system('less --quit-if-one-screen %s' % tempfile.name)
  739
+
  740
+
  741
+#
  742
+# Code to make this run as an executable from the command line.
  743
+#
  744
+
  745
+class _UsageException(Exception): pass
  746
+
  747
+def _parse_command_line():
  748
+  import getopt, sys, re
  749
+  try:
  750
+    opts, remaining_args = getopt.gnu_getopt(
  751
+        sys.argv[1:], 'md:',
  752
+        ['start=', 'end=', 'extensions=', 'editor=', 'count', 'test'])
  753
+  except getopt.error:
  754
+    raise _UsageException()
  755
+  opts = dict(opts)
  756
+
  757
+  if '--test' in opts:
  758
+    import doctest
  759
+    doctest.testmod()
  760
+    sys.exit(0)
  761
+
  762
+  query_options = {}
  763
+  if len(remaining_args) in [1, 2]:
  764
+    query_options['suggestor'] = (
  765
+     (multiline_regex_suggestor if '-m' in opts else regex_suggestor)
  766
+     (*remaining_args)  # remaining_args is [regex] or [regex, substitution].
  767
+    )
  768
+  else:
  769
+    raise _UsageException()
  770
+  if '--start' in opts:
  771
+    query_options['start'] = opts['--start']
  772
+  if '--end' in opts:
  773
+    query_options['end'] = opts['--end']
  774
+  if '-d' in opts:
  775
+    query_options['root_directory'] = opts['-d']
  776
+  if '--extensions' in opts:
  777
+    query_options['path_filter'] = (
  778
+        path_filter(extensions=opts['--extensions'].split(',')))
  779
+
  780
+  options = {}
  781
+  options['query'] = Query(**query_options)
  782
+  if '--editor' in opts:
  783
+    options['editor'] = opts['--editor']
  784
+  if '--count' in opts:
  785
+    options['just_count'] = True
  786
+
  787
+  return options
  788
+
  789
+if __name__ == '__main__':
  790
+  try:
  791
+    options = _parse_command_line()
  792
+  except _UsageException:
  793
+    print_through_less(__doc__.strip())
  794
+    sys.exit(2)
  795
+  run_interactive(**options)

1 note on commit 8da5577

andrew02

https://andrew02@github.com/andrew02/codemod.git

Please sign in to comment.
Something went wrong with that request. Please try again.