Permalink
Switch branches/tags
Find file
Fetching contributors…
Cannot retrieve contributors at this time
905 lines (731 sloc) 34.6 KB
''' Tratihubis converts Trac tickets to Github issues by using the
following steps:
1. The user manually exports the Trac tickets to convert to a CSV file.
2. Tratihubis reads the CSV file and uses the data to create Github issues and milestones.
Installation
============
To install tratihubis, use ``pip`` or ``easy_install``::
$ pip install tratihubis
If necessary, this also installs the `PyGithub <http://pypi.python.org/pypi/PyGithub/>`_ package.
Usage
=====
Information about Trac tickets to convert has to be provided in a CSV file. To obtain this CSV file, create a
new Trac queries using the SQL statement stored in
`query_tickets.sql <https://github.com/roskakori/tratihubis/blob/master/query_tickets.sql>`_ and
`query_comments.sql <https://github.com/roskakori/tratihubis/blob/master/query_comments.sql>`_. Then
execute the queries and save the results by clicking "Download in other formats: Comma-delimited Text" and
choosing for example ``/Users/me/mytool/tickets.csv`` and ``/Users/me/mytool/comments.csv`` as output files.
Next create a config file to describe how to login to Github and what to convert. For example, you could
store the following in ``~/mytool/tratihubis.cfg``::
[tratihubis]
token = my_github_token
repo = mytool
tickets = /Users/me/mytool/tickets.csv
comments = /Users/me/mytool/comments.csv
Then run::
$ tratihubis ~/mytool/tratihubis.cfg
This tests that the input data and Github information is valid and writes a log to the console describing
which operations would be performed.
To actually create the Github issues, you need to enable to command line option ``--really``::
$ tratihubis --really ~/mytool/tratihubis.cfg
Be aware that Github issues and milestones cannot be deleted in case you mess up. Your only remedy is to
remove the whole repository and start anew. So make sure that tratihubis does what you want before you
enable ``--really``.
Mapping users
-------------
By default all tickets and comments are created by the user specified with the option `token`. For
private Trac project with a single user this already gives the desired result in the Github project.
In case there are multiple Trac users, you can map them to different Github tokens using the option
`users`. For example::
users = johndoe: johndoe_token, *: another_token, sally: *
This maps the Trac user ``ohndoe` using John Doe's Github token and everyone else to
`another_token`. Sally is mapped to default token as specified with the `token` option above,
which in this example is `my_github_token`.
The default value is::
users = *:*
This maps every Trac user to the default token.
Mapping labels
--------------
Github labels somewhat mimic the functionality Trac stores in the ``type`` and ``resolution`` field of
tickets. By default, Github supports the following labels:
* bug
* duplicate
* enhancement
* invalid
* question
* wontfix
Trac on the other hand has a ``type`` field which by default can be:
* bug
* enhancement
* task
Furthermore closed Trac tickets have a ``resolution`` which, among others, can be:
* duplicate
* invalid
* wontfix
The ``labels`` config option allows to map Trac fields to Github labels. For example::
labels = type=defect: bug, type=enhancement: enhancement, resolution=wontfix: wontfix
Here, ``labels`` is a comma separated list of mappings taking the form
``<trac-field>=<trac-value>:<github-label>``.
In case types or labels contain other characters than ASCII letters, digits and underscore (_), put them
between quotes::
labels = type="software defect": bug
Attachments
-----------
You can find some notes on this in `issue #19 <https://github.com/roskakori/tratihubis/issues/19>`: Add
documentation for ``attachmentsprefix``.
Converting Trac Wiki Markup to Github Markdown
----------------------------------------------
Tratihubis makes an attempt to convert Trac Wiki markup into
Github markdown with the use of a number of regular expression
substitutions. This is far from a perfect process so care should be
taken and the the regular expressions may need amending on a case by
case basis. This conversion process also tries to preserve links
between tickets with expressions such as `ticket:XX` converted to
`issue #YY`. Also links in Trac such as `rXXXX` when referring to
subversion changeset will link back to the original Trac repository if
required. Use::
convert_text = true
in the `.cfg` file to attempt the markup conversion. Stipulate the
Trac repository with::
trac_url = https://trac/url
Limitations
===========
The author of Github issues and comments always is the user specified in the config, even if a different
user opened the original Trac ticket or wrote the original Trac comment.
Github issues and comments have the current time as time stamp instead if time from Trac.
Github issue descriptions contains the raw Trac Wiki markup, there is no translation to Github markdown.
The due date of Trac milestones is not migrated to Github milestones, so when the conversion is done, you
have to set it manually.
Trac Milestone without any tickets are not converted to Github milestone.
Support
=======
In case of questions and problems, open an issue at <https://github.com/roskakori/tratihubis/issues>.
To obtain the source code or create your own fork to implement fixes or improvements, visit
<https://github.com/roskakori/tratihubis>.
License
=======
Copyright (c) 2012-2013, Thomas Aglassinger. All rights reserved. Distributed under the
`BSD License <http://www.opensource.org/licenses/bsd-license.php>`_.
Changes
=======
Version 1.0, 2014-06-14
(Contributed by Daniel Wheeler)
* Changed user authentication from password to token.
* Added basic translation of Wiki markup.
* Added conversion of ticket:xx links.
* Added backlink to from Github issue to original Trac link.
Version 0.5, 2013-02-13
(Contributed by Steven Di Rocco)
* Added support for file attachments.
* Added work around for information lost due GitHub API limitations:
* Added trac commenter and date at the top of each comment.
* Added automatic comment to each isseu showing original author, date authored, and last modification date.
* Fixed API calls to work with PyGithub 1.8.
Version 0.4, 2012-05-04
* Added config option ``labels`` to map Trac status and resolution to Github labels.
Version 0.3, 2012-05-03
* Added config option ``comments`` to convert Trac ticket comments.
* Added closing of issue for which the corresponding Trac ticket has been closed already.
* Added validation of users issues are assigned to. They must have an active Github user.
Version 0.2, 2012-05-02
* Added config option ``users`` to map Trac users to Github users.
* Added binary in order to run ``tratihubis`` instead of ``python -m tratihubis``.
* Changed supposed issue number in log to take existing issues in account.
Version 0.1, 2012-05-01
* Initial release.
'''
# Copyright (c) 2012-2013, Thomas Aglassinger
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# * Neither the name of Thomas Aglassinger nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
import codecs
import collections
import ConfigParser
import csv
import github
import logging
import optparse
import os.path
import StringIO
import sys
import token
import tokenize
import datetime
from translator import Translator, NullTranslator
_log = logging.getLogger('tratihubis')
__version__ = "1.0"
_SECTION = 'tratihubis'
_OPTION_LABELS = 'labels'
_OPTION_USERS = 'users'
_validatedGithubTokens = set()
_FakeMilestone = collections.namedtuple('_FakeMilestone', ['number', 'title'])
_FakeIssue = collections.namedtuple('_FakeIssue', ['number', 'title', 'body', 'state'])
csv.field_size_limit(sys.maxsize)
class _ConfigError(Exception):
def __init__(self, option, message):
assert option is not None
assert message is not None
Exception.__init__(self, u'cannot process config option "%s" in section [%s]: %s'
% (option, _SECTION, message))
class _CsvDataError(Exception):
def __init__(self, csvPath, rowIndex, message):
assert csvPath is not None
assert rowIndex is not None
assert rowIndex >= 0
assert message is not None
Exception.__init__(self, u'%s:%d: %s' % (os.path.basename(csvPath), rowIndex + 1, message))
class _UTF8Recoder:
"""
Iterator that reads an encoded stream and reencodes the input to UTF-8
"""
def __init__(self, f, encoding):
self.reader = codecs.getreader(encoding)(f)
def __iter__(self):
return self
def next(self): # @ReservedAssignment
result = self.reader.next().encode("utf-8")
return result
class _UnicodeCsvReader:
"""
A CSV reader which will iterate over lines in the CSV file "f",
which is encoded in the given encoding.
"""
def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
f = _UTF8Recoder(f, encoding)
self.reader = csv.reader(f, dialect=dialect, **kwds)
def next(self): # @ReservedAssignment
row = self.reader.next()
return [unicode(s, "utf-8") for s in row]
def __iter__(self):
return self
class _LabelTransformations(object):
def __init__(self, repo, definition):
assert repo is not None
self._transformations = []
self._labelMap = {}
if definition:
self._buildLabelMap(repo)
self._buildTransformations(repo, definition)
def _buildLabelMap(self, repo):
assert repo is not None
_log.info(u'analyze existing labels')
self._labelMap = {}
for label in repo.get_labels():
_log.debug(u' found label "%s"', label.name)
self._labelMap[label.name] = label
_log.info(u' found %d labels', len(self._labelMap))
def _buildTransformations(self, repo, definition):
assert repo is not None
assert definition is not None
STATE_AT_TRAC_FIELD = 'f'
STATE_AT_COMPARISON_OPERATOR = '='
STATE_AT_TRAC_VALUE = 'v'
STATE_AT_COLON = ':'
STATE_AT_LABEL = 'l'
STATE_AT_COMMA = ','
self._transformations = []
state = STATE_AT_TRAC_FIELD
for tokenType, tokenText, _, _, _ in tokenize.generate_tokens(StringIO.StringIO(definition).readline):
if tokenType == token.STRING:
tokenText = tokenText[1:len(tokenText) - 1]
if state == STATE_AT_TRAC_FIELD:
tracField = tokenText
tracValue = None
labelValue = None
state = STATE_AT_COMPARISON_OPERATOR
elif state == STATE_AT_COMPARISON_OPERATOR:
if tokenText != '=':
raise _ConfigError(_OPTION_LABELS,
u'Trac field "%s" must be followed by \'=\' instead of %r'
% (tracField, tokenText))
state = STATE_AT_TRAC_VALUE
elif state == STATE_AT_TRAC_VALUE:
tracValue = tokenText
state = STATE_AT_COLON
elif state == STATE_AT_COLON:
if tokenText != ':':
raise _ConfigError(_OPTION_LABELS,
u'value for comparison "%s" with Trac field "%s" must be followed by \':\' instead of %r'
% (tracValue, tracField, tokenText))
state = STATE_AT_LABEL
elif state == STATE_AT_LABEL:
labelValue = tokenText
if not labelValue in self._labelMap:
raise _ConfigError(_OPTION_LABELS,
u'unknown label "%s" must be replaced by one of: %s'
% (labelValue, sorted(self._labelMap.keys())))
self._transformations.append((tracField, tracValue, labelValue))
state = STATE_AT_COMMA
elif state == STATE_AT_COMMA:
if (tokenType != token.ENDMARKER) and (tokenText != ','):
raise _ConfigError(_OPTION_LABELS,
u'label transformation for Trac field "%s" must end with \',\' instead of %r'
% (tracField, tokenText))
state = STATE_AT_TRAC_FIELD
else:
assert False, u'state=%r' % state
def labelFor(self, tracField, tracValue):
assert tracField
assert tracValue is not None
result = None
transformationIndex = 0
while (result is None) and (transformationIndex < len(self._transformations)):
transformedField, transformedValueToCompareWith, transformedLabel = \
self._transformations[transformationIndex]
if (transformedField == tracField) and (transformedValueToCompareWith == tracValue):
assert transformedLabel in self._labelMap
result = self._labelMap[transformedLabel]
else:
transformationIndex += 1
return result
def _getConfigOption(config, name, required=True, defaultValue=None, boolean=False):
try:
if boolean:
result = config.getboolean(_SECTION, name)
else:
result = config.get(_SECTION, name)
except ConfigParser.NoOptionError:
if required:
raise _ConfigError(name, 'config must contain a value for this option')
result = defaultValue
except ConfigParser.NoSectionError:
raise _ConfigError(name, u'config must contain this section')
return result
def _shortened(text):
assert text is not None
THRESHOLD = 30
if len(text) > THRESHOLD:
result = text[:THRESHOLD] + '...'
else:
result = text
return result
def _addNewLabel(label, repo):
if label not in [l.name for l in repo.get_labels()]:
repo.create_label(label, '5319e7')
def _tracTicketMaps(ticketsCsvPath):
"""
Sequence of maps where each items describes the relevant fields of each row from the tickets CSV exported
from Trac.
"""
EXPECTED_COLUMN_COUNT = 12
_log.info(u'read ticket details from "%s"', ticketsCsvPath)
with open(ticketsCsvPath, "rb") as ticketCsvFile:
csvReader = _UnicodeCsvReader(ticketCsvFile)
hasReadHeader = True
for rowIndex, row in enumerate(csvReader):
columnCount = len(row)
if columnCount != EXPECTED_COLUMN_COUNT:
raise _CsvDataError(ticketsCsvPath, rowIndex,
u'ticket row must have %d columns but has %d: %r' %
(EXPECTED_COLUMN_COUNT, columnCount, row))
if hasReadHeader:
ticketMap = {
'id': long(row[0]),
'type': row[1],
'owner': row[2],
'reporter': row[3],
'milestone': row[4],
'status': row[5],
'resolution': row[6],
'summary': row[7],
'description': row[8],
'createdtime': datetime.datetime.fromtimestamp(long(row[9])),
'modifiedtime': datetime.datetime.fromtimestamp(long(row[10])),
'component': row[11]
}
yield ticketMap
else:
hasReadHeader = True
def _createMilestoneMap(repo):
def addMilestones(targetMap, state):
for milestone in repo.get_milestones(state=state):
_log.debug(u' %d: %s', milestone.number, milestone.title)
targetMap[milestone.title] = milestone
result = {}
_log.info(u'analyze existing milestones')
addMilestones(result, 'open')
addMilestones(result, 'closed')
_log.info(u' found %d milestones', len(result))
return result
def _createIssueMap(repo):
def addIssues(targetMap, state):
for issue in repo.get_issues(state=state):
_log.debug(u' %s: (%s) %s', issue.number, issue.state, issue.title)
targetMap[issue.number] = issue
result = {}
_log.info(u'analyze existing issues')
addIssues(result, 'open')
addIssues(result, 'closed')
_log.info(u' found %d issues', len(result))
return result
def _createTicketToCommentsMap(commentsCsvPath):
EXPECTED_COLUMN_COUNT = 4
result = {}
if commentsCsvPath is not None:
_log.info(u'read ticket comments from "%s"', commentsCsvPath)
with open(commentsCsvPath, "rb") as commentsCsvFile:
csvReader = _UnicodeCsvReader(commentsCsvFile)
hasReadHeader = True
for rowIndex, row in enumerate(csvReader):
columnCount = len(row)
if columnCount != EXPECTED_COLUMN_COUNT:
raise _CsvDataError(commentsCsvPath, rowIndex,
u'comment row must have %d columns but has %d: %r' %
(EXPECTED_COLUMN_COUNT, columnCount, row))
if hasReadHeader:
commentMap = {
'id': long(row[0]),
'date': datetime.datetime.fromtimestamp(long(row[1])),
'author': row[2],
'body': row[3],
}
ticketId = commentMap['id']
ticketComments = result.get(ticketId)
if ticketComments is None:
ticketComments = []
result[ticketId] = ticketComments
ticketComments.append(commentMap)
else:
hasReadHeader = True
return result
def is_int(s):
try:
long(s)
return True
except ValueError:
return False
def _createTicketsToAttachmentsMap(attachmentsCsvPath, attachmentsPrefix):
EXPECTED_COLUMN_COUNT = 4
result = {}
if attachmentsCsvPath is not None and attachmentsPrefix is None:
_log.error(u'attachments csv path specified but attachmentsprefix is not\n')
return result
if attachmentsCsvPath is not None:
_log.info(u'read attachments from "%s"', attachmentsCsvPath)
else:
return result
with open(attachmentsCsvPath, "rb") as attachmentsCsvFile:
attachmentsReader = _UnicodeCsvReader(attachmentsCsvFile)
hasReadHeader = True
for rowIndex, row in enumerate(attachmentsReader):
columnCount = len(row)
if columnCount != EXPECTED_COLUMN_COUNT:
raise _CsvDataError(attachmentsCsvPath, rowIndex,
u'attachment row must have %d columns but has %d: %r' %
(EXPECTED_COLUMN_COUNT, columnCount, row))
if hasReadHeader:
id_string = row[0]
if is_int(id_string):
attachmentMap = {
'id': long(id_string),
'author': row[3],
'filename': row[1],
'date': datetime.datetime.fromtimestamp(long(row[2])),
'fullpath': u'%s/%s/%s' % (attachmentsPrefix, row[0], row[1]),
}
if not attachmentMap['id'] in result:
result[attachmentMap['id']] = [attachmentMap]
else:
result[attachmentMap['id']].append(attachmentMap)
else:
hasReadHeader = True
return result
def createTicketsToIssuesMap(ticketsCsvPath, existingIssues, firstTicketIdToConvert, lastTicketIdToConvert):
ticketsToIssuesMap = dict()
fakeIssueId = 1 + len(existingIssues)
for ticketMap in _tracTicketMaps(ticketsCsvPath):
ticketId = ticketMap['id']
if (ticketId >= firstTicketIdToConvert) \
and ((ticketId <= lastTicketIdToConvert) or (lastTicketIdToConvert == 0)):
ticketsToIssuesMap[int(ticketId)] = fakeIssueId
fakeIssueId += 1
return ticketsToIssuesMap
def migrateTickets(hub, repo, defaultToken, ticketsCsvPath,
commentsCsvPath=None, attachmentsCsvPath=None,
firstTicketIdToConvert=1, lastTicketIdToConvert=0,
labelMapping=None, userMapping="*:*",
attachmentsPrefix=None, pretend=True,
trac_url=None, convert_text=False, ticketsToRender=False, addComponentLabels=False):
assert hub is not None
assert repo is not None
assert ticketsCsvPath is not None
assert userMapping is not None
tracTicketToCommentsMap = _createTicketToCommentsMap(commentsCsvPath)
tracTicketToAttachmentsMap = _createTicketsToAttachmentsMap(attachmentsCsvPath, attachmentsPrefix)
existingIssues = _createIssueMap(repo)
existingMilestones = _createMilestoneMap(repo)
tracToGithubUserMap = _createTracToGithubUserMap(hub, userMapping, defaultToken)
labelTransformations = _LabelTransformations(repo, labelMapping)
ticketsToIssuesMap = createTicketsToIssuesMap(ticketsCsvPath, existingIssues, firstTicketIdToConvert, lastTicketIdToConvert)
if convert_text:
Translator_ = Translator
else:
Translator_ = NullTranslator
translator = Translator_(repo, ticketsToIssuesMap, trac_url=trac_url, attachmentsPrefix=attachmentsPrefix)
def possiblyAddLabel(labels, tracField, tracValue):
label = labelTransformations.labelFor(tracField, tracValue)
if label is not None:
_log.info(' add label %s', label.name)
if not pretend:
labels.append(label.name)
fakeIssueId = 1 + len(existingIssues)
for ticketMap in _tracTicketMaps(ticketsCsvPath):
ticketId = ticketMap['id']
title = ticketMap['summary']
renderTicket = True
if ticketsToRender:
if not ticketId in ticketsToRender:
renderTicket = False
if renderTicket and (ticketId >= firstTicketIdToConvert) \
and ((ticketId <= lastTicketIdToConvert) or (lastTicketIdToConvert == 0)):
body = ticketMap['description']
tracOwner = ticketMap['reporter'].strip()
token = _tokenFor(hub, tracToGithubUserMap, tracOwner)
_hub = github.Github(token)
_repo = _hub.get_repo('{0}/{1}'.format(repo.owner.login, repo.name))
githubAssignee = _hub.get_user()
milestoneTitle = ticketMap['milestone'].strip()
if len(milestoneTitle) != 0:
if milestoneTitle not in existingMilestones:
_log.info(u'add milestone: %s', milestoneTitle)
print existingMilestones
if not pretend:
newMilestone = repo.create_milestone(milestoneTitle)
else:
newMilestone = _FakeMilestone(len(existingMilestones) + 1, milestoneTitle)
existingMilestones[milestoneTitle] = newMilestone
milestone = existingMilestones[milestoneTitle]
milestoneNumber = milestone.number
else:
milestone = None
milestoneNumber = 0
_log.info(u'convert ticket #%d: %s', ticketId, _shortened(title))
title = translator.translate(title)
body = translator.translate(body, ticketId=ticketId)
dateformat = "%m-%d-%Y at %H:%M"
ticketString = '#{0}'.format(ticketId)
if trac_url:
ticket_url = '/'.join([trac_url, 'ticket', str(ticketId)])
ticketString = '[{0}]({1})'.format(ticketString, ticket_url)
legacyInfo = u"\n\n _Imported from trac ticket %s, created by %s on %s, last modified: %s_\n" \
% (ticketString, ticketMap['reporter'], ticketMap['createdtime'].strftime(dateformat),
ticketMap['modifiedtime'].strftime(dateformat))
body += legacyInfo
if ticketsToRender:
print 'body of ticket:\n', body
if not pretend:
if milestone is None:
issue = _repo.create_issue(title, body)#, githubAssignee)
else:
issue = _repo.create_issue(title, body, milestone=milestone)
else:
issue = _FakeIssue(fakeIssueId, title, body, 'open')
fakeIssueId += 1
_log.info(u' issue #%s: owner=%s-->%s; milestone=%s (%d)',
issue.number, tracOwner, githubAssignee.name, milestoneTitle, milestoneNumber)
labels = []
possiblyAddLabel(labels, 'type', ticketMap['type'])
possiblyAddLabel(labels, 'resolution', ticketMap['resolution'])
if addComponentLabels and ticketMap['component'] != 'None':
if not pretend:
labels.append(ticketMap['component'])
if not pretend:
for l in labels:
_addNewLabel(l, repo)
if len(labels) > 0:
_hub = github.Github(defaultToken)
_repo = _hub.get_repo('{0}/{1}'.format(repo.owner.login, repo.name))
_issue = _repo.get_issue(issue.number)
_issue.edit(labels=labels)
attachmentsToAdd = tracTicketToAttachmentsMap.get(ticketId)
if attachmentsToAdd is not None:
for attachment in attachmentsToAdd:
token = _tokenFor(repo, tracToGithubUserMap, attachment['author'], False)
attachmentAuthor = _userFor(token)
legacyInfo = u"_%s attached [%s](%s) on %s_\n" \
% (attachment['author'], attachment['filename'], attachment['fullpath'], attachment['date'].strftime(dateformat))
_log.info(u' added attachment from %s', attachmentAuthor)
if ticketsToRender:
print 'attachment legacy info:\n',legacyInfo
if not pretend:
assert issue is not None
issue.create_comment(legacyInfo)
commentsToAdd = tracTicketToCommentsMap.get(ticketId)
if commentsToAdd is not None:
for comment in commentsToAdd:
token = _tokenFor(repo, tracToGithubUserMap, comment['author'], False)
commentAuthor = _userFor(token)
_hub = github.Github(token)
_repo = _hub.get_repo('{0}/{1}'.format(repo.owner.login, repo.name))
commentBody = u"%s\n\n_Trac comment by %s on %s_\n" % (comment['body'], comment['author'], comment['date'].strftime(dateformat))
_log.info(u' add comment by %s: %r', commentAuthor, _shortened(commentBody))
commentBody = translator.translate(commentBody, ticketId=ticketId)
if ticketsToRender:
print 'commentBody:\n',commentBody
if not pretend:
_issue = _repo.get_issue(issue.number)
assert _issue is not None
_issue.create_comment(commentBody)
if ticketMap['status'] == 'closed':
_log.info(u' close issue')
if not pretend:
issue.edit(state='closed')
else:
_log.info(u'skip ticket #%d: %s', ticketId, title)
def _parsedOptions(arguments):
assert arguments is not None
# Parse command line options.
Usage = 'usage: %prog [options] CONFIGFILE\n\n Convert Trac tickets to Github issues.'
parser = optparse.OptionParser(
usage=Usage,
version="%prog " + __version__
)
parser.add_option("-R", "--really", action="store_true", dest="really",
help="really perform the conversion")
parser.add_option("-v", "--verbose", action="store_true", dest="verbose",
help="log all actions performed in console")
(options, others) = parser.parse_args(arguments)
if len(others) == 0:
parser.error(u"CONFIGFILE must be specified")
elif len(others) > 1:
parser.error(u"unknown options must be removed: %s" % others[1:])
if options.verbose:
_log.setLevel(logging.DEBUG)
configPath = others[0]
return options, configPath
def _validateGithubUser(hub, tracUser, token):
assert hub is not None
assert tracUser is not None
assert token is not None
if token not in _validatedGithubTokens:
try:
_log.debug(u' check for token "%s"', token)
_hub = github.Github(token)
githubUser = _hub.get_user()
_log.debug(u' user is "%s"',)
except:
# FIXME: After PyGithub API raises a predictable error, use "except WahteverException".
raise _ConfigError(_OPTION_USERS,
u'Trac user "%s" must be mapped to an existing Github users token instead of "%s"'
% (tracUser, githubUser))
_validatedGithubTokens.add(token)
def _createTracToGithubUserMap(hub, definition, defaultToken):
result = {}
for mapping in definition.split(','):
words = [word.strip() for word in mapping.split(':')]
if words:
if len(words) != 2:
raise _ConfigError(_OPTION_USERS,
u'mapping must use syntax "trac-user: token" but is: "%s"' % mapping)
tracUser, token = words
if token == '*':
token = defaultToken
# if len(tracUser) == 0:
# raise _ConfigError(_OPTION_USERS, u'Trac user must not be empty: "%s"' % mapping)
if len(token) == 0:
raise _ConfigError(_OPTION_USERS, u'Token not be empty: "%s"' % mapping)
existingMappedGithubUser = result.get(tracUser)
if existingMappedGithubUser is not None:
raise _ConfigError(_OPTION_USERS,
u'Trac user "%s" must be mapped to only one token instead of "%s" and "%s"'
% (tracUser, existingMappedGithubUser, token))
result[tracUser] = token
if token != '*':
_validateGithubUser(hub, tracUser, token)
return result
def _tokenFor(hub, tracToGithubUserMap, tracUser, validate=True):
assert tracToGithubUserMap is not None
assert tracUser is not None
result = tracToGithubUserMap.get(tracUser)
if result is None:
result = tracToGithubUserMap.get('*')
if result is None:
raise _ConfigError(_OPTION_USERS, u'Trac user "%s" must be mapped to a Github user' % (tracUser,))
if result == '*':
result = tracUser
if validate:
_validateGithubUser(hub, tracUser, result)
return result
def _userFor(token):
_hub = github.Github(token)
return _hub.get_user()
def main(argv=None):
if argv is None:
argv = sys.argv
exitCode = 1
try:
options, configPath = _parsedOptions(argv[1:])
config = ConfigParser.SafeConfigParser()
config.read(configPath)
commentsCsvPath = _getConfigOption(config, 'comments', False)
attachmentsCsvPath = _getConfigOption(config, 'attachments', False)
attachmentsPrefix = _getConfigOption(config, 'attachmentsprefix', False)
labelMapping = _getConfigOption(config, 'labels', False)
repoName = _getConfigOption(config, 'repo')
ticketsCsvPath = _getConfigOption(config, 'tickets', False, 'tickets.csv')
token = _getConfigOption(config, 'token')
userMapping = _getConfigOption(config, 'users', False, '*:{0}'.format(token))
trac_url = _getConfigOption(config, 'trac_url', False)
convert_text = _getConfigOption(config, 'convert_text',
required=False,
defaultValue=False,
boolean=True)
ticketsToRender = _getConfigOption(config,
'ticketsToRender',
required=False,
defaultValue=False,
boolean=False)
addComponentLabels = _getConfigOption(config, 'addComponentLabels',
required=False,
defaultValue=False,
boolean=True)
if ticketsToRender:
ticketsToRender = [long(x) for x in ticketsToRender.split(',')]
if not options.really:
_log.warning(u'no actions are performed unless command line option --really is specified')
hub = github.Github(token)
_log.info(u'log on to github as user "%s"', hub.get_user().login)
repo = hub.get_user().get_repo(repoName)
_log.info(u'connect to github repo "%s"', repoName)
migrateTickets(hub, repo, token, ticketsCsvPath,
commentsCsvPath, attachmentsCsvPath,
userMapping=userMapping,
labelMapping=labelMapping,
attachmentsPrefix=attachmentsPrefix,
pretend=not options.really,
trac_url=trac_url, convert_text=convert_text, ticketsToRender=ticketsToRender, addComponentLabels=addComponentLabels)
exitCode = 0
except (EnvironmentError, OSError, _ConfigError, _CsvDataError), error:
_log.error(error)
except KeyboardInterrupt:
_log.warning(u"interrupted by user")
except Exception, error:
_log.exception(error)
return exitCode
def _mainEntryPoint():
logging.basicConfig(level=logging.INFO)
sys.exit(main())
if __name__ == "__main__":
_mainEntryPoint()