Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

SVN completed #179

Merged
merged 1 commit into from over 2 years ago

3 participants

Pradeepkumar Gayam Dustin J. Mitchell Tom Prince
Pradeepkumar Gayam
in3xes commented July 11, 2011

Please take closer look than usual, especially purge and url stuff.

master/docs/cfg-buildsteps.texinfo
((190 lines not shown))
  851
+  @{'name':'test-ppc', 'slavename':'bot-ppc', 'builddir':'test-ppc',
  852
+                      'factory':f @},
  853
+ ]
  854
+@end example
  855
+
  856
+In this example, when a change arrives with a @code{branch} attribute of
  857
+``trunk'', the resulting build will have a SVN step that concatenates
  858
+``svn://svn.example.org/MyProject/'' (the baseURL) with ``trunk'' (the branch
  859
+name) to get the correct svn command. If the ``newthing'' branch has a change
  860
+to ``src/foo.c'', then the SVN step will concatenate
  861
+``svn://svn.example.org/MyProject/'' with ``features/newthing'' to get the
  862
+svnurl for checkout.
  863
+
  864
+For added flexibility, @code{baseURL} may contain a @code{%%BRANCH%%}
  865
+placeholder, which will be replaced either by the branch in the SourceStamp or
  866
+the default specified in @code{defaultBranch}.
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

This should be described in the @item for baseURL, above!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/docs/cfg-buildsteps.texinfo
((76 lines not shown))
  737
+@item depth
  738
+(optional): Specify depth argument to achieve sparse checkout.  Only available if slave has Subversion 1.5 or higher.
  739
+
  740
+If set to "empty" updates will not pull in any files or subdirectories not already present. If set to "files", updates will pull in any files not already present, but not directories.  If set to "immediates", updates will pull in any files or subdirectories not already present, the new subdirectories will have depth: empty.  If set to "infinity", updates will pull in any files or subdirectories not already present; the new subdirectories will have depth-infinity. Infinity is equivalent to SVN default update behavior, without specifying any depth argument.
  741
+
  742
+@item mode
  743
+@item method
  744
+
  745
+SVN's incremental mode does not require a method.  The full mode has
  746
+four methods defined:
  747
+
  748
+@table @code
  749
+@item clobber
  750
+It removes the working directory for each build then makes full checkout.
  751
+@item fresh
  752
+This is alternative for incremental (old update) mode with @code{always_purge}
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

Better phrased as "This is equivalent to the old update mode with @code{always_purge}", and only mentioned as the last sentence in this @item. That is:

This always always purges local changes before updating. This deletes
unversioned files and reverts everything that would appear in a @code{svn status
--no-ignore}. This is equivalent to the old update mode with @code{always_purge}.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff July 20, 2011
master/buildbot/test/unit/test_steps_source_svn.py
((390 lines not shown))
  390
+                        command=['svn', 'update', '--non-interactive',
  391
+                                 '--no-auth-cache'])
  392
+            + 0,
  393
+            ExpectLogged('cpdir', {'fromdir': 'source',
  394
+                                   'todir': 'build'})
  395
+            + 0,
  396
+            ExpectShell(workdir='build',
  397
+                        command=['svnversion'])
  398
+            + ExpectShell.log('stdio',
  399
+                stdout='100')
  400
+            + 0,
  401
+        )
  402
+        self.expectOutcome(result=SUCCESS, status_text=["update"])
  403
+        return self.runStep()
  404
+
  405
+class TestGetUnversionedFiles(unittest.TestCase):
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

I'd like to see some more tests here - invalid XML, an XML document with no entries, an entry with no wc-status, a status with no item, and a wc-status with no path, at least -- just to see how those failures are handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((325 lines not shown))
  325
+        return c
  326
+
  327
+    @staticmethod
  328
+    def getUnversionedFiles(xmlStr, keep_on_purge):
  329
+        """Delete everything that shown up on status."""
  330
+        result_xml = parseString(xmlStr)
  331
+        for entry in result_xml.getElementsByTagName('entry'):
  332
+            (wc_status,) = entry.getElementsByTagName('wc-status')
  333
+            if wc_status.getAttribute('item') == 'external':
  334
+                continue
  335
+            if wc_status.getAttribute('item') == 'missing':
  336
+                continue
  337
+            filename = entry.getAttribute('path')
  338
+            if filename in keep_on_purge:
  339
+                continue
  340
+            yield filename
2
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

Nice use of a generator :)

Pradeepkumar Gayam
in3xes added a note July 22, 2011

I copied that from slaveside ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((314 lines not shown))
  314
+                    cmd.useLog(self.stdio_log, False)
  315
+                    d = self.runCommand(cmd)
  316
+                    d.addCallback(lambda _: cmd.rc)
  317
+            return d
  318
+        c.addCallback(parseAndRemove)
  319
+        def evaluateCommand(rc):
  320
+            if rc != 0:
  321
+                log.msg("Failed removing files")
  322
+                raise failure.Failure(rc)
  323
+            return rc
  324
+        c.addCallback(evaluateCommand)
  325
+        return c
  326
+
  327
+    @staticmethod
  328
+    def getUnversionedFiles(xmlStr, keep_on_purge):
  329
+        """Delete everything that shown up on status."""
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

This comment isn't quite right.. this method doesn't delete anything!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff July 20, 2011
master/buildbot/steps/source/svn.py
((312 lines not shown))
  312
+                else:
  313
+                    cmd = buildstep.LoggedRemoteCommand('rmdir', {'dir': files})
  314
+                    cmd.useLog(self.stdio_log, False)
  315
+                    d = self.runCommand(cmd)
  316
+                    d.addCallback(lambda _: cmd.rc)
  317
+            return d
  318
+        c.addCallback(parseAndRemove)
  319
+        def evaluateCommand(rc):
  320
+            if rc != 0:
  321
+                log.msg("Failed removing files")
  322
+                raise failure.Failure(rc)
  323
+            return rc
  324
+        c.addCallback(evaluateCommand)
  325
+        return c
  326
+
  327
+    @staticmethod
2
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

Why static?

Pradeepkumar Gayam
in3xes added a note July 22, 2011

Only for tests, no other reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell djmitche commented on the diff July 20, 2011
master/buildbot/steps/source/svn.py
((287 lines not shown))
  287
+                raise
  288
+
  289
+            log.msg("Got SVN revision %s" % (revision, ))
  290
+            self.setProperty('got_revision', revision, 'Source')
  291
+            return res
  292
+        d.addCallback(lambda _: _setrev(cmd.rc))
  293
+        return d
  294
+
  295
+    def purge(self, ignore_ignores):
  296
+        command = ['status', '--xml']
  297
+        if ignore_ignores:
  298
+            command.append('--no-ignore')
  299
+        c = self._dovccmd(command)
  300
+        def parseAndRemove(_):
  301
+            output = self.getLog('stdio').getText()
  302
+            output = output[output.find('<'):]
2
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

I'm not sure what this is about - finding the first tag? Why?

Pradeepkumar Gayam
in3xes added a note July 22, 2011

When I do self.getLog('stdio').getText() it gives the output previous commands concatenated to one after other. So, I am find first '<' to find the starting of the xml data. Any other way?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((84 lines not shown))
  84
+                                 depth=depth,
  85
+                                 )
  86
+
  87
+        assert self.mode in ['incremental', 'full']
  88
+        assert self.method in ['clean', 'fresh', 'clobber', 'copy', None]
  89
+
  90
+        if svnurl and baseURL:
  91
+            raise ValueError("you must provide exactly one of svnurl and"
  92
+                             " baseURL")
  93
+
  94
+        if svnurl is None and baseURL is None:
  95
+            raise ValueError("you must privide at least one of svnurl and"
  96
+                             " baseURL")
  97
+
  98
+        self.svnurl = self.svnurl and _ComputeRepositoryURL(self.svnurl)
  99
+        self.baseURL = self.baseURL and _ComputeRepositoryURL(self.baseURL)
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

Since you submitted this pull req, @tomprince has removed the _ComputeRepositoryURL invocations from new source steps, so please remove them here, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((58 lines not shown))
  58
+        @param username: username to pass to svn's --username
  59
+
  60
+        @type  password: string
  61
+        @param password: password to pass to svn's --password
  62
+        """
  63
+
  64
+        self.svnurl = svnurl
  65
+        self.baseURL = baseURL
  66
+        self.branch = defaultBranch
  67
+        self.username = username
  68
+        self.password = password
  69
+        self.extra_args = extra_args
  70
+        self.keep_on_purge = keep_on_purge or []
  71
+        self.depth = depth
  72
+        self.method=method
  73
+        Source.__init__(self, **kwargs)
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

This is an unusual place to call the parent constructor. It should either be called before the other attributes are set, or after all of them are set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
master/buildbot/steps/source/svn.py
((23 lines not shown))
  23
+from buildbot.interfaces import BuildSlaveTooOldError
  24
+
  25
+
  26
+class SVN(Source):
  27
+    """I perform Subversion checkout/update operations."""
  28
+
  29
+    name = 'svn'
  30
+    branch_placeholder = '%BRANCH%'
  31
+
  32
+    renderables = [ 'svnurl', 'baseURL' ]
  33
+
  34
+    def __init__(self, svnurl=None, baseURL=None, mode='incremental',
  35
+                 method=None, defaultBranch=None, username=None,
  36
+                 password=None, extra_args=None, keep_on_purge=None,
  37
+                 depth=None, **kwargs):
  38
+        """
1
Dustin J. Mitchell Owner
djmitche added a note July 20, 2011

This docstring duplicates information available in the manual, and in fact doesn't even cover all of the constructor arguments. I think it should be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Dustin J. Mitchell
Owner

A few points:

baseURL vs svnurl

These names are confusing, and as I understand it, are just a vestige of backward compatibility. Since this step requires that users choose it explicitly, we have a chance to do it right. What do these two parameters accomplish, and how could we do that with one parameter, with lots of flexibility for the user? What about the common case when the repository in the SourceStamp is sufficient - do we need to make the user specify a URL to the SVN step in that case, too? You don't need to implement the full flexibility yet, but think ahead when deciding how to handle these URLs.

Tests on master

I pulled this onto master, and while it merged smoothly, the tests did not pass after the merge. This is probably "bitrot" after it took me so long to review - sorry.. Can you take a look?

Tom Prince
Collaborator
Pradeepkumar Gayam
in3xes commented July 23, 2011

Updated, will fix URL stuff later.

Dustin J. Mitchell
Owner

Merged (finally), yay!

Dustin J. Mitchell djmitche merged commit a31ef36 into from July 26, 2011
Dustin J. Mitchell djmitche closed this July 26, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Jul 23, 2011
Pradeepkumar Gayam SVN completed a31ef36
Something went wrong with that request. Please try again.