failure when using Id token

[spoore1]

While looking at #65 I looked at the betelgeuse code and I noticed in add_test_case:

371         # Fetch the test case id if the @Id tag is present otherwise generate a
372         # test_case_id based on the test Python import path

So, I thought I'd try to use that to set the test case "title" in polarion:

# cat test_spoore_betelgeuse_01.py 
class IpaBetelgeuseExampleClass(object):
    def test_betelgeuse_example_0001(self):
        """ @Id: IDM-IPA-TC: IPA Betelgeuse Example 0001
            @Description: New test 0001
        """
        pass

But, I see this error:

# betelgeuse test-case --path test_spoore_betelgeuse_01.py RedHatEnterpriseLinux7
Updating test case test_betelgeuse_example_0001 for requirement Spoore Betelgeuse 01.
Traceback (most recent call last):
  File "/usr/bin/betelgeuse", line 9, in <module>
    load_entry_point('Betelgeuse==0.4.0', 'console_scripts', 'betelgeuse')()
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/click-6.6-py2.7.egg/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "build/bdist.linux-x86_64/egg/betelgeuse.py", line 625, in test_case
    pool.join()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
AssertionError

That error, I think is related to the test_case_id including spaces. I added a simple patch to my local copy to add quotes:

diff --git a/betelgeuse.py b/betelgeuse.py
index 3c2a035..c0f4d86 100644
--- a/betelgeuse.py
+++ b/betelgeuse.py
@@ -414,6 +414,7 @@ def add_test_case(args):

         results = []
         if not collect_only:
+            test_case_id = '"{}"'.format(test_case_id)
             results = TestCase.query(
                 test_case_id,
                 fields=[

That seemed to resolve the issue I was seeing.

But, it appears as if the test case was still created with the method name instead of the 'Id' token value.

[spoore1]

FYI, I coupled that fix above with another slight modification here:

spoore1@6f37fd1

Not sure if this is something we want long term since you're working on other solutions. But, FYI, I was able to get it to import cases using the Id token as the test title that way.

[omaciel]

hmmmm we treat title and id as 2 different, distinct parameters, where the id is a unique field to help us locate a test case. The title however would be the first line found in the docstring:

class IpaBetelgeuseExampleClass(object):
    def test_betelgeuse_example_0001(self):
        """IDM-IPA-TC: IPA Betelgeuse Example 0001
            @Id: IDM-IPA-TC: IPA Betelgeuse Example 0001
            @Description: New test 0001
        """
        pass

[spoore1]

I need to set the polarion test case "title" which I think is actually accessed via pylarion as the test case "name" right?

And, yeah, when I add Id token entry to first line, the @id: is included in the title as well. Is the testimony title imported to polarion in any way for test cases?

FYI, this case was really just meant to show that error and adding quotes around the test_case_id. Not sure if you guys wanted me to post a pull request for that or wait to see if other changes in the pipe would resolve that.

[omaciel]

@spoore1 send in a PR and let's have a discussion about it. It is true that the polarion title is derived from the test case name (which would be the name of your python test method). You want to specify a different name instead, right? Like @Title: Foo bar.

[elyezer]

I think if @title is present use that otherwise use the test method name would be a good resolution.

[spoore1]

Yes, I want to specify the polarion test case name somewhere per case. At the moment, my "patch" just allows me to use Id token but, I am by no means opposed to using Title. That makes more sense I think but, that doesn't work now does it?

I just saw the comment about @id and thought I'd try that.

[spoore1]

@omaciel I created a PR with both the test_case_id quoting and new changes to allow using Title token for test.name per suggestion from @elyezer . I did have a slight deviation where I let it use Title first, Id if no Title, and then function if neither.

I guess at this point, this thread can be closed and discussion can move to that PR?

[elyezer]

@spoore1 let's close this when the PR is merged. I will take some time and review it.

[elyezer]

I was not able to reproduce the exception, I am using latest Betelgeuse, Testimony and Pylarion.

I've tried the following:

    def test_something(self):
        """@ID: id with spaces

        @description: Description of the test
        """

    def test_something(self):
        """@ID: IDM-IPA-TC: IPA Betelgeuse Example 0001
        @description: Description of the test
        """

    def test_something(self):
        """IDM-IPA-TC: IPA Betelgeuse Example 0001
            @ID: IDM-IPA-TC: IPA Betelgeuse Example 0001
            @description: Description of the test
        """

All the above attempts worked well.

Looking at the bottom of the stack trace we can see that there is an assertion error and the only place we use assert o Betelgeuse test is https://github.com/SatelliteQE/betelgeuse/blob/master/betelgeuse.py#L467-L469. That means that Polarion query is returning more than one test case, probably because the spaces which explains the need of the quotes since Polarion will try to match on every word.

I went ahead on my investigation and tried the following:

    def test_something(self):
        """IDM-IPA-TC: IPA Betelgeuse Example 0001

        @ID: IDM-IPA-TC: IPA Betelgeuse Example 0001
        @description: Description of the test
        """

    def test_something_1(self):
        """IDM-IPA-TC: IPA Betelgeuse Example 0002

        @ID: IDM-IPA-TC: IPA Betelgeuse Example 0002
        """

Trying to make Polarion return more than one result and I was not able to raise any exception. My conclusion is that on the Project you are trying there is another test case which is being returned by the query for that specific ID, I recommend you trying to search on the UI for that ID and see what are the test cases returned so you can delete and adjust.

Using an unique ID is not a Betelgeuse's responsibility and the recommended way is to use uuid.uuid4() in order to generate the IDs, that will ensure the uniqueness and is more fail proof than other approaches.

I will go ahead and close this issue since it is not a Betelguese bug. I have created #73 which will make Betelgeuse output useful information when more than one test case are returned.