Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

SVN completed #179

Merged
merged 1 commit into from

3 participants

Pradeepkumar Gayam Dustin J. Mitchell Tom Prince
Pradeepkumar Gayam

Please take closer look than usual, especially purge and url stuff.

master/docs/cfg-buildsteps.texinfo
((190 lines not shown))
+ @{'name':'test-ppc', 'slavename':'bot-ppc', 'builddir':'test-ppc',
+ 'factory':f @},
+ ]
+@end example
+
+In this example, when a change arrives with a @code{branch} attribute of
+``trunk'', the resulting build will have a SVN step that concatenates
+``svn://svn.example.org/MyProject/'' (the baseURL) with ``trunk'' (the branch
+name) to get the correct svn command. If the ``newthing'' branch has a change
+to ``src/foo.c'', then the SVN step will concatenate
+``svn://svn.example.org/MyProject/'' with ``features/newthing'' to get the
+svnurl for checkout.
+
+For added flexibility, @code{baseURL} may contain a @code{%%BRANCH%%}
+placeholder, which will be replaced either by the branch in the SourceStamp or
+the default specified in @code{defaultBranch}.
Dustin J. Mitchell Owner

This should be described in the @item for baseURL, above!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/docs/cfg-buildsteps.texinfo
((76 lines not shown))
+@item depth
+(optional): Specify depth argument to achieve sparse checkout. Only available if slave has Subversion 1.5 or higher.
+
+If set to "empty" updates will not pull in any files or subdirectories not already present. If set to "files", updates will pull in any files not already present, but not directories. If set to "immediates", updates will pull in any files or subdirectories not already present, the new subdirectories will have depth: empty. If set to "infinity", updates will pull in any files or subdirectories not already present; the new subdirectories will have depth-infinity. Infinity is equivalent to SVN default update behavior, without specifying any depth argument.
+
+@item mode
+@item method
+
+SVN's incremental mode does not require a method. The full mode has
+four methods defined:
+
+@table @code
+@item clobber
+It removes the working directory for each build then makes full checkout.
+@item fresh
+This is alternative for incremental (old update) mode with @code{always_purge}
Dustin J. Mitchell Owner

Better phrased as "This is equivalent to the old update mode with @code{always_purge}", and only mentioned as the last sentence in this @item. That is:

This always always purges local changes before updating. This deletes
unversioned files and reverts everything that would appear in a @code{svn status
--no-ignore}. This is equivalent to the old update mode with @code{always_purge}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff
master/buildbot/test/unit/test_steps_source_svn.py
((390 lines not shown))
+ command=['svn', 'update', '--non-interactive',
+ '--no-auth-cache'])
+ + 0,
+ ExpectLogged('cpdir', {'fromdir': 'source',
+ 'todir': 'build'})
+ + 0,
+ ExpectShell(workdir='build',
+ command=['svnversion'])
+ + ExpectShell.log('stdio',
+ stdout='100')
+ + 0,
+ )
+ self.expectOutcome(result=SUCCESS, status_text=["update"])
+ return self.runStep()
+
+class TestGetUnversionedFiles(unittest.TestCase):
Dustin J. Mitchell Owner

I'd like to see some more tests here - invalid XML, an XML document with no entries, an entry with no wc-status, a status with no item, and a wc-status with no path, at least -- just to see how those failures are handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((325 lines not shown))
+ return c
+
+ @staticmethod
+ def getUnversionedFiles(xmlStr, keep_on_purge):
+ """Delete everything that shown up on status."""
+ result_xml = parseString(xmlStr)
+ for entry in result_xml.getElementsByTagName('entry'):
+ (wc_status,) = entry.getElementsByTagName('wc-status')
+ if wc_status.getAttribute('item') == 'external':
+ continue
+ if wc_status.getAttribute('item') == 'missing':
+ continue
+ filename = entry.getAttribute('path')
+ if filename in keep_on_purge:
+ continue
+ yield filename
Dustin J. Mitchell Owner

Nice use of a generator :)

Pradeepkumar Gayam
in3xes added a note

I copied that from slaveside ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((314 lines not shown))
+ cmd.useLog(self.stdio_log, False)
+ d = self.runCommand(cmd)
+ d.addCallback(lambda _: cmd.rc)
+ return d
+ c.addCallback(parseAndRemove)
+ def evaluateCommand(rc):
+ if rc != 0:
+ log.msg("Failed removing files")
+ raise failure.Failure(rc)
+ return rc
+ c.addCallback(evaluateCommand)
+ return c
+
+ @staticmethod
+ def getUnversionedFiles(xmlStr, keep_on_purge):
+ """Delete everything that shown up on status."""
Dustin J. Mitchell Owner

This comment isn't quite right.. this method doesn't delete anything!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff
master/buildbot/steps/source/svn.py
((312 lines not shown))
+ else:
+ cmd = buildstep.LoggedRemoteCommand('rmdir', {'dir': files})
+ cmd.useLog(self.stdio_log, False)
+ d = self.runCommand(cmd)
+ d.addCallback(lambda _: cmd.rc)
+ return d
+ c.addCallback(parseAndRemove)
+ def evaluateCommand(rc):
+ if rc != 0:
+ log.msg("Failed removing files")
+ raise failure.Failure(rc)
+ return rc
+ c.addCallback(evaluateCommand)
+ return c
+
+ @staticmethod
Dustin J. Mitchell Owner

Why static?

Pradeepkumar Gayam
in3xes added a note

Only for tests, no other reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff
master/buildbot/steps/source/svn.py
((287 lines not shown))
+ raise
+
+ log.msg("Got SVN revision %s" % (revision, ))
+ self.setProperty('got_revision', revision, 'Source')
+ return res
+ d.addCallback(lambda _: _setrev(cmd.rc))
+ return d
+
+ def purge(self, ignore_ignores):
+ command = ['status', '--xml']
+ if ignore_ignores:
+ command.append('--no-ignore')
+ c = self._dovccmd(command)
+ def parseAndRemove(_):
+ output = self.getLog('stdio').getText()
+ output = output[output.find('<'):]
Dustin J. Mitchell Owner

I'm not sure what this is about - finding the first tag? Why?

Pradeepkumar Gayam
in3xes added a note

When I do self.getLog('stdio').getText() it gives the output previous commands concatenated to one after other. So, I am find first '<' to find the starting of the xml data. Any other way?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((84 lines not shown))
+ depth=depth,
+ )
+
+ assert self.mode in ['incremental', 'full']
+ assert self.method in ['clean', 'fresh', 'clobber', 'copy', None]
+
+ if svnurl and baseURL:
+ raise ValueError("you must provide exactly one of svnurl and"
+ " baseURL")
+
+ if svnurl is None and baseURL is None:
+ raise ValueError("you must privide at least one of svnurl and"
+ " baseURL")
+
+ self.svnurl = self.svnurl and _ComputeRepositoryURL(self.svnurl)
+ self.baseURL = self.baseURL and _ComputeRepositoryURL(self.baseURL)
Dustin J. Mitchell Owner

Since you submitted this pull req, @tomprince has removed the _ComputeRepositoryURL invocations from new source steps, so please remove them here, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((58 lines not shown))
+ @param username: username to pass to svn's --username
+
+ @type password: string
+ @param password: password to pass to svn's --password
+ """
+
+ self.svnurl = svnurl
+ self.baseURL = baseURL
+ self.branch = defaultBranch
+ self.username = username
+ self.password = password
+ self.extra_args = extra_args
+ self.keep_on_purge = keep_on_purge or []
+ self.depth = depth
+ self.method=method
+ Source.__init__(self, **kwargs)
Dustin J. Mitchell Owner

This is an unusual place to call the parent constructor. It should either be called before the other attributes are set, or after all of them are set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((23 lines not shown))
+from buildbot.interfaces import BuildSlaveTooOldError
+
+
+class SVN(Source):
+ """I perform Subversion checkout/update operations."""
+
+ name = 'svn'
+ branch_placeholder = '%BRANCH%'
+
+ renderables = [ 'svnurl', 'baseURL' ]
+
+ def __init__(self, svnurl=None, baseURL=None, mode='incremental',
+ method=None, defaultBranch=None, username=None,
+ password=None, extra_args=None, keep_on_purge=None,
+ depth=None, **kwargs):
+ """
Dustin J. Mitchell Owner

This docstring duplicates information available in the manual, and in fact doesn't even cover all of the constructor arguments. I think it should be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell
Owner

A few points:

baseURL vs svnurl

These names are confusing, and as I understand it, are just a vestige of backward compatibility. Since this step requires that users choose it explicitly, we have a chance to do it right. What do these two parameters accomplish, and how could we do that with one parameter, with lots of flexibility for the user? What about the common case when the repository in the SourceStamp is sufficient - do we need to make the user specify a URL to the SVN step in that case, too? You don't need to implement the full flexibility yet, but think ahead when deciding how to handle these URLs.

Tests on master

I pulled this onto master, and while it merged smoothly, the tests did not pass after the merge. This is probably "bitrot" after it took me so long to review - sorry.. Can you take a look?

Tom Prince
Owner
Pradeepkumar Gayam

Updated, will fix URL stuff later.

Dustin J. Mitchell
Owner

Merged (finally), yay!

Dustin J. Mitchell djmitche merged commit a31ef36 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 23, 2011
  1. Pradeepkumar Gayam

    SVN completed

    in3xes authored
Something went wrong with that request. Please try again.