Getting exception creating Rally object with 1.0.0

[patknight]

I'm trying to create the initial Rally object and I'm getting the following traceback. The same call works fine with 0.9.4. If you let me know what else you need to debug or anything you need me to run that will help, I'll try to get that for you.

rally=Rally(RALLY_SERVER,RALLY_USER,RALLY_PASSWORD,workspace=RALLY_WORKSPACE,project=RALLY_PROJECT)
ERROR: Unable to access item 10 (9 items served so far from a container purported to be 13 items in length)
Traceback (most recent call last):
File "", line 1, in
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/restapi.py", line 223, in init
self.contextHelper.check(self.server)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/context.py", line 226, in check
self._getDefaults(response)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/context.py", line 293, in _getDefaults
projects = self.agent.get('Project', fetch="Name", workspace=self._defaultWorkspace)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/restapi.py", line 786, in get
context, augments = self.contextHelper.identifyContext(**kwargs)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/context.py", line 543, in identifyContext
self._establishContext(kwargs)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/context.py", line 522, in _establishContext
self._getWorkspacesAndProjects(workspace=workspace, project=project)
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/context.py", line 638, in _getWorkspacesAndProjects
for project in response:
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/rallyresp.py", line 271, in next
self._page[:] = self.__retrieveNextPage()
File "/Users/pknight/Library/Python/2.7/lib/python/site-packages/pyral/rallyresp.py", line 327, in __retrieveNextPage
content = json.loads(response.content)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 383, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

The same call works fine with 0.9.4. What o

[chrisking]

I'm having the same issue as well with version "pyral==1.0.0"

ERROR: Unable to access item 10 (9 items served so far from a container purported to be 13 items in length)
Traceback (most recent call last):
  File "hardware_manager.py", line 31, in <module>
    project="Lab Infrastructure")
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/restapi.py", line 223, in __init__
    self.contextHelper.check(self.server)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/context.py", line 226, in check
    self._getDefaults(response)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/context.py", line 293, in _getDefaults
    projects = self.agent.get('Project', fetch="Name", workspace=self._defaultWorkspace)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/restapi.py", line 786, in get
    context, augments = self.contextHelper.identifyContext(**kwargs)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/context.py", line 543, in identifyContext
    self._establishContext(kwargs)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/context.py", line 522, in _establishContext
    self._getWorkspacesAndProjects(workspace=workspace, project=project)
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/context.py", line 638, in _getWorkspacesAndProjects
    for project in response:
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/rallyresp.py", line 271, in next
    self._page[:] = self.__retrieveNextPage()
  File "/home/ubuntu/.virtualenvs/rally_lab/local/lib/python2.7/site-packages/pyral/rallyresp.py", line 327, in __retrieveNextPage
    content = json.loads(response.content)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

[klehman-rally]

It appears that the response to REST API request to retrieve the project collection is returning less than what is advertised. If there was a branch here in the repo that exposed via a log file the specific project collection urls used, would you be willing to throw those urls into something like a browser REST client and observe the results? Using a JSON prettifier might make eyeballing the output more palatable. The aim of this would be to determine whether the url is constructed correctly and the response is "shorted" or if somehow the constructed url is subtly incorrect. I'm guessing your subscription has more workspaces / projects than the ones I was using for testing, but it'd probably be illuminating to take some specifics back to the Rally ALM dev team about how a response says it has a TotalResultCount of 13 but actually contains only 9 in the response data.

[patknight]

It's been a busy day, so I'm only just getting to this. The workspace I'm using in the call has 22 projects in it, with some nested in others. So I don't know where the 9 or the 13 are coming from :-)

As you suggest, I'll use a browser REST client. What should the URL look like, or will I find that documented on the Rally site?

[klehman-rally]

Pat, the project collection url will look something like https://rally1rallydev.com/slm/webservice/v2.0/Workspace/<workspace_ObjectID_value>/Projects?fetch="ObjectID,Name,State"?pagesize=200&start=1 where the specific workspace_ObjectID_value will be context dependent. If you can easily determine the offending Workspace, you should be able to plug in the OID for it in this project collection url. Perusing our own defect list, I do see some prior mention of actual results not matching TotalResultCount, but the defect that looked most appropriate was fixed and released late last year. There's others that allude to the prospect of user permissions having a role. Is the user account you are using a Subscription Admin or a Workspace Admin or a regular user?

[klehman-rally]

Pat, looks like the Markdown took out the interesting part of the url given as an example. The specific workspace object ID value should be inserted between the two consecutive slashes (Workspace slash workspace_object_ID slash Projects).

[patknight]

We have two workspaces and during setup, pyral retrieves both. I'd assumed it was retrieving the workspace I passed to the constructor and that's the one I looked at in Rally to count projects, getting 22. So don't worry about that.

I used Postman in Chrome with the URL https://rally1.rallydev.com/slm/webservice/v2.0/workspace/*workspace-OID* as that's what was used at the point pyral did the exception. Here's the project count for the other workspace coming back as 13 in the response. If I log into the Rally GUI and count projects for that workspace in the workspaces and projects page, I count 9.

If I then go to the workspace's details page and look at the children, there are 13! There are 4 closed projects. So what's going on is Rally is telling you the total project count but only returning the open ones.

 "Projects": {
            "_rallyAPIMajor": "2",
            "_rallyAPIMinor": "0",
            "_ref": "https://rally1.rallydev.com/slm/webservice/v2.0/Workspace/*******/Projects",
            "_type": "Project",
            "Count": 13
        }

[patknight]

Hi - since I gathered the information in the previous comment, there's been no update. Do you need further information from me?

[klehman-rally]

Digging around in our own product issue tracking info led me to find that we do have a WSAPI defect filed for this. Unfortunately the priority has been been downgraded at least once so a fix is not "imminent".
How would the following approach work for you...? Special casing the setup where the projects for each workspace are retrieved is "protected" in a try/except so that if/when the WSAPI fibs about the number of project items it has returned doesn't result in the ignominius death of the code. It passes back to the caller the projects it was successfully able to obtain information for. It runs the risk of encountering a workspace in which no projects are classified non 'Closed'. See any problems or gotchas for your situation?

[patknight]

I thought it sounded like a WSAPI issue. I'd be happy with your proposal about catching the exception in this case - the Internet philosophy of be strict in what you send, tolerant (within reason) in what you'll receive. HTTP content sizes should let you know if anything went missing when the count comes up short (i.e. there really were 13 open projects and we only got data for 9) so pyral shouldn't end up silently ignoring errors.

If a workspace has all projects closed, I don't think that should be a special case - there'll be zero projects we can do anything with (and people will just have to use the GUI to manipulate closed projects - that's not anything your client can do anything about.)

Is the WSAPI defect public? If so, I could "express interest" in it to add my tiny weight to increasing its priority.

And by the way, thanks for the effort you put into this project. Although there are deadlines at our end so I might push a little, in the end I do remember this free (as in cost) software and that it's not your problem if those deadlines are missed.

[klehman-rally]

Pat and Chris, would you be willing to pass along the name of the organization with the Rally subscription? As our tracking tool doesn't have a public "interest indicator" option, I'd like to hoist the wand signaling "customer has great need" towards the folks who make WSAPI prioritization decisions :-) It's possible the needle hasn't moved recently because there's not be anyone asking for the needle to be moved. It can't hurt.

[patknight]

If you could give me an email address so it's not in public, I'm willing. Note, however, that Chris and I (and a couple of others) are the only serious scripting users at the moment - everyone else uses the GUI.

[klehman-rally]

Pat, you can contact me at klehman@rallydev.com . And your scripting seriousness is important to to us! Don't short sell it before we attempt to squeeze some effort out of the other side of the organism.

[patknight]

I was hit by the issue this morning in a production script (my fault for not reverting to 0.9.4 after testing.) I've done the following patch as a workaround for now (it works for our use cases) to raise StopIteration if we fall off the end of the list of projects.

*** rallyresp.py~   Fri May 16 09:46:24 2014
--- rallyresp.py    Wed May 28 12:53:08 2014
***************
*** 267,273 ****
                  verbiage = "Unable to access item %d (%d items served so far from a " +\
                             "container purported to be %d items in length)"
                  problem = verbiage % (self._curIndex+1, self._served, self.resultCount)
!                 errout("ERROR: %s\n" % problem)
                  self._page[:] = self.__retrieveNextPage()
                  pprint(self._page[0])
                  raise IndexError("RallyRESTResponse._page[%d]" % self._curIndex)
--- 267,274 ----
                  verbiage = "Unable to access item %d (%d items served so far from a " +\
                             "container purported to be %d items in length)"
                  problem = verbiage % (self._curIndex+1, self._served, self.resultCount)
!                 errout("WARNING: %s\n" % problem)
!                 raise StopIteration
                  self._page[:] = self.__retrieveNextPage()
                  pprint(self._page[0])
                  raise IndexError("RallyRESTResponse._page[%d]" % self._curIndex)

[ssbarnea]

What is happening with this bug?

The python API is completely broken and nobody confirms working on it.

I tried to repair the raise by changing it to RaiseIteration() but it fails later. After using the debugger I discovered that I get:

422 The specified workspace is not accessible.

I tried both ways, with / without specifying the workspace name still getting the same error which is not even identified properly.

[patknight]

Read the thread. It seems obvious from the interactions I've had with klehman that he's working on this.

Look at my patch from Wednesday and check the error message you're getting before changing the released code at all. Make sure you're encountering the same issue. With that patch, my automation started working.