Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

SVN completed #179

Merged
merged 1 commit into from

3 participants

@in3xes

Please take closer look than usual, especially purge and url stuff.

master/docs/cfg-buildsteps.texinfo
((190 lines not shown))
+ @{'name':'test-ppc', 'slavename':'bot-ppc', 'builddir':'test-ppc',
+ 'factory':f @},
+ ]
+@end example
+
+In this example, when a change arrives with a @code{branch} attribute of
+``trunk'', the resulting build will have a SVN step that concatenates
+``svn://svn.example.org/MyProject/'' (the baseURL) with ``trunk'' (the branch
+name) to get the correct svn command. If the ``newthing'' branch has a change
+to ``src/foo.c'', then the SVN step will concatenate
+``svn://svn.example.org/MyProject/'' with ``features/newthing'' to get the
+svnurl for checkout.
+
+For added flexibility, @code{baseURL} may contain a @code{%%BRANCH%%}
+placeholder, which will be replaced either by the branch in the SourceStamp or
+the default specified in @code{defaultBranch}.
@djmitche Owner

This should be described in the @item for baseURL, above!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/docs/cfg-buildsteps.texinfo
((76 lines not shown))
+@item depth
+(optional): Specify depth argument to achieve sparse checkout. Only available if slave has Subversion 1.5 or higher.
+
+If set to "empty" updates will not pull in any files or subdirectories not already present. If set to "files", updates will pull in any files not already present, but not directories. If set to "immediates", updates will pull in any files or subdirectories not already present, the new subdirectories will have depth: empty. If set to "infinity", updates will pull in any files or subdirectories not already present; the new subdirectories will have depth-infinity. Infinity is equivalent to SVN default update behavior, without specifying any depth argument.
+
+@item mode
+@item method
+
+SVN's incremental mode does not require a method. The full mode has
+four methods defined:
+
+@table @code
+@item clobber
+It removes the working directory for each build then makes full checkout.
+@item fresh
+This is alternative for incremental (old update) mode with @code{always_purge}
@djmitche Owner

Better phrased as "This is equivalent to the old update mode with @code{always_purge}", and only mentioned as the last sentence in this @item. That is:

This always always purges local changes before updating. This deletes
unversioned files and reverts everything that would appear in a @code{svn status
--no-ignore}. This is equivalent to the old update mode with @code{always_purge}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@djmitche djmitche commented on the diff
master/buildbot/test/unit/test_steps_source_svn.py
((390 lines not shown))
+ command=['svn', 'update', '--non-interactive',
+ '--no-auth-cache'])
+ + 0,
+ ExpectLogged('cpdir', {'fromdir': 'source',
+ 'todir': 'build'})
+ + 0,
+ ExpectShell(workdir='build',
+ command=['svnversion'])
+ + ExpectShell.log('stdio',
+ stdout='100')
+ + 0,
+ )
+ self.expectOutcome(result=SUCCESS, status_text=["update"])
+ return self.runStep()
+
+class TestGetUnversionedFiles(unittest.TestCase):
@djmitche Owner

I'd like to see some more tests here - invalid XML, an XML document with no entries, an entry with no wc-status, a status with no item, and a wc-status with no path, at least -- just to see how those failures are handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((325 lines not shown))
+ return c
+
+ @staticmethod
+ def getUnversionedFiles(xmlStr, keep_on_purge):
+ """Delete everything that shown up on status."""
+ result_xml = parseString(xmlStr)
+ for entry in result_xml.getElementsByTagName('entry'):
+ (wc_status,) = entry.getElementsByTagName('wc-status')
+ if wc_status.getAttribute('item') == 'external':
+ continue
+ if wc_status.getAttribute('item') == 'missing':
+ continue
+ filename = entry.getAttribute('path')
+ if filename in keep_on_purge:
+ continue
+ yield filename
@djmitche Owner

Nice use of a generator :)

@in3xes
in3xes added a note

I copied that from slaveside ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((314 lines not shown))
+ cmd.useLog(self.stdio_log, False)
+ d = self.runCommand(cmd)
+ d.addCallback(lambda _: cmd.rc)
+ return d
+ c.addCallback(parseAndRemove)
+ def evaluateCommand(rc):
+ if rc != 0:
+ log.msg("Failed removing files")
+ raise failure.Failure(rc)
+ return rc
+ c.addCallback(evaluateCommand)
+ return c
+
+ @staticmethod
+ def getUnversionedFiles(xmlStr, keep_on_purge):
+ """Delete everything that shown up on status."""
@djmitche Owner

This comment isn't quite right.. this method doesn't delete anything!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@djmitche djmitche commented on the diff
master/buildbot/steps/source/svn.py
((312 lines not shown))
+ else:
+ cmd = buildstep.LoggedRemoteCommand('rmdir', {'dir': files})
+ cmd.useLog(self.stdio_log, False)
+ d = self.runCommand(cmd)
+ d.addCallback(lambda _: cmd.rc)
+ return d
+ c.addCallback(parseAndRemove)
+ def evaluateCommand(rc):
+ if rc != 0:
+ log.msg("Failed removing files")
+ raise failure.Failure(rc)
+ return rc
+ c.addCallback(evaluateCommand)
+ return c
+
+ @staticmethod
@djmitche Owner

Why static?

@in3xes
in3xes added a note

Only for tests, no other reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@djmitche djmitche commented on the diff
master/buildbot/steps/source/svn.py
((287 lines not shown))
+ raise
+
+ log.msg("Got SVN revision %s" % (revision, ))
+ self.setProperty('got_revision', revision, 'Source')
+ return res
+ d.addCallback(lambda _: _setrev(cmd.rc))
+ return d
+
+ def purge(self, ignore_ignores):
+ command = ['status', '--xml']
+ if ignore_ignores:
+ command.append('--no-ignore')
+ c = self._dovccmd(command)
+ def parseAndRemove(_):
+ output = self.getLog('stdio').getText()
+ output = output[output.find('<'):]
@djmitche Owner

I'm not sure what this is about - finding the first tag? Why?

@in3xes
in3xes added a note

When I do self.getLog('stdio').getText() it gives the output previous commands concatenated to one after other. So, I am find first '<' to find the starting of the xml data. Any other way?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((84 lines not shown))
+ depth=depth,
+ )
+
+ assert self.mode in ['incremental', 'full']
+ assert self.method in ['clean', 'fresh', 'clobber', 'copy', None]
+
+ if svnurl and baseURL:
+ raise ValueError("you must provide exactly one of svnurl and"
+ " baseURL")
+
+ if svnurl is None and baseURL is None:
+ raise ValueError("you must privide at least one of svnurl and"
+ " baseURL")
+
+ self.svnurl = self.svnurl and _ComputeRepositoryURL(self.svnurl)
+ self.baseURL = self.baseURL and _ComputeRepositoryURL(self.baseURL)
@djmitche Owner

Since you submitted this pull req, @tomprince has removed the _ComputeRepositoryURL invocations from new source steps, so please remove them here, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((58 lines not shown))
+ @param username: username to pass to svn's --username
+
+ @type password: string
+ @param password: password to pass to svn's --password
+ """
+
+ self.svnurl = svnurl
+ self.baseURL = baseURL
+ self.branch = defaultBranch
+ self.username = username
+ self.password = password
+ self.extra_args = extra_args
+ self.keep_on_purge = keep_on_purge or []
+ self.depth = depth
+ self.method=method
+ Source.__init__(self, **kwargs)
@djmitche Owner

This is an unusual place to call the parent constructor. It should either be called before the other attributes are set, or after all of them are set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((23 lines not shown))
+from buildbot.interfaces import BuildSlaveTooOldError
+
+
+class SVN(Source):
+ """I perform Subversion checkout/update operations."""
+
+ name = 'svn'
+ branch_placeholder = '%BRANCH%'
+
+ renderables = [ 'svnurl', 'baseURL' ]
+
+ def __init__(self, svnurl=None, baseURL=None, mode='incremental',
+ method=None, defaultBranch=None, username=None,
+ password=None, extra_args=None, keep_on_purge=None,
+ depth=None, **kwargs):
+ """
@djmitche Owner

This docstring duplicates information available in the manual, and in fact doesn't even cover all of the constructor arguments. I think it should be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@djmitche
Owner

A few points:

baseURL vs svnurl

These names are confusing, and as I understand it, are just a vestige of backward compatibility. Since this step requires that users choose it explicitly, we have a chance to do it right. What do these two parameters accomplish, and how could we do that with one parameter, with lots of flexibility for the user? What about the common case when the repository in the SourceStamp is sufficient - do we need to make the user specify a URL to the SVN step in that case, too? You don't need to implement the full flexibility yet, but think ahead when deciding how to handle these URLs.

Tests on master

I pulled this onto master, and while it merged smoothly, the tests did not pass after the merge. This is probably "bitrot" after it took me so long to review - sorry.. Can you take a look?

@tomprince
Owner
@in3xes

Updated, will fix URL stuff later.

@djmitche
Owner

Merged (finally), yay!

@djmitche djmitche merged commit a31ef36 into buildbot:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 23, 2011
  1. @in3xes

    SVN completed

    in3xes authored
Something went wrong with that request. Please try again.