New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Behaviour of a test page B which is a child of a test page A? #538

Merged
merged 2 commits into from Oct 31, 2014

Conversation

Projects
None yet
4 participants
@amolenaar
Collaborator

amolenaar commented Oct 31, 2014

A test page "B" is the child of test page "A". If "A" is tested than always "B" gets tested as well.

So "A" behaves like a suite page.

Not sure if this is a bug or a feature.
But I am quit sure this behaviour was different in older versions.

The behaviour can stay like this for me. But want to be sure that it will stay like this before I start using it.

@ALuckyGuy

This comment has been minimized.

ALuckyGuy commented Oct 17, 2014

I noticed this recently myself. Seems like a Test page shouldn't demonstrate suite behavior, otherwise why make a distinction at all.

@amolenaar

This comment has been minimized.

Collaborator

amolenaar commented Oct 21, 2014

In the last release both ?suite and ?test respond to the same responder. Their behaviour was basically identical. I did not consider the case of nesting test pages (what's the use case for that?).

@ALuckyGuy

This comment has been minimized.

ALuckyGuy commented Oct 22, 2014

I can think of a few reasons for it, though I'm sure people might have different reasons.

I've done this to save older versions of the test page that I don't necessarily want to delete. Saving these older versions in the primary pages directory keeps them organized and grouped together, regardless of what long and descriptive name I might give the different versions.

Additionally, I've had symbolic links on my test page that I didn't want to execute from my test page but just want a convenient means to link to parts of my wiki.

Finally, I created a test page that's a test runner, configuration variables defined on the page and a table containing the pages to execute along with email addresses and other info that indicates what to do with the results. How I hoped it would work is when executing it would create symlinks to the pages it would run and then call them restfully. That works, but the problem is the test page executes the symlinked page just like a suite, so in the end I had to create a child page of the symlinks and have that page set to "skip" so the test page would ignore it. That shouldn't have been necessary if the Test page simply execute it's page contents alone.

By the way, I'm seeing this behavior in the Aug 25th build, so it's not that recent. But if you're telling me older versions of Test pages will in fact ignore their child pages, except SetUp and TearDown of course, I might actually go back to an older release for a while because it is a bit of an annoyance. The distinction between Test pages and Suite pages is useful.

@six42

This comment has been minimized.

Contributor

six42 commented Oct 26, 2014

The use case I see is to add the main functionality of a requirement on one page and the exceptional cases on child pages. http concordion.org uses this approach.
It has the advantage that a reader knows what are the key requirements/features and reads these first. He might skip reading the details of the exceptional cases if he just wants to get an overview of the available features.

Thinking about is:

As a requirements author and reader I want to group related requirements together and want to define the order in which they are displayed. As in a book pages which are at the same level in the page hierarchy should have the same importance or level of detail. Or in other words the page hierarchy should match the table of content if I would write a normal paper based specification.
To ensure that each page is focused and easy to understand I want the description of a single requirement or feature be written on multiple pages.

As a developer and tester I want to be able to test a single piece (page) of requirement and be able to test all related requirements in one step. If I test all related requirements the order in which the results are displayed should match the order defined by the author and all additional text and images given by the author should be in the results. I want a history of the results in both cases (testing a single page or a set of pages)

@six42

This comment has been minimized.

Contributor

six42 commented Oct 30, 2014

Forget to add a summary: I would vote to role back the change which introduced the current behaviour.

@amolenaar amolenaar added this to the Next release milestone Oct 30, 2014

@amolenaar

This comment has been minimized.

Collaborator

amolenaar commented Oct 30, 2014

The generic/specific test case scenario you describe makes sense. I'll add a few lines to get the old ?test behaviour back.

@amolenaar

This comment has been minimized.

Collaborator

amolenaar commented Oct 31, 2014

The attached patch should do the trick :)

@amolenaar amolenaar merged commit 49c3a8f into unclebob:master Oct 31, 2014

amolenaar added a commit that referenced this pull request Oct 31, 2014

Merge pull request #538 from amolenaar/testresponder
Behaviour of a test page B which is a child of a test page A?

@amolenaar amolenaar deleted the amolenaar:testresponder branch Nov 7, 2014

@ALuckyGuy

This comment has been minimized.

ALuckyGuy commented Nov 18, 2014

This doesn't seem quite fixed to me. Or at least it's not behaving how I expected.

When a Test page A has a child test page B, executing Test A alone does in fact ignore the child B. Great. However what was surprising is that executing the Suite that Test A is a part of, child B is executed. I don't think this should occur. If I had a sub-suite that didn't execute a page, the parent suite won't either. Like-wise, I would think if executing a Test A doesn't execute the child, executing Test A as part of a suite should ignore Test A's child too. Basically a Test page should be a leaf node when it comes to execution and a suite shouldn't go looking for children beneath the leaf.

If the current behavior matches prior behavior before the "In the last release both ?suite and ?test respond to the same responder" change, then I guess it should be preserved and I didn't notice it previously. But if this doesn't match the prior behavior, it should be rolled back too.

@amolenaar

This comment has been minimized.

Collaborator

amolenaar commented Nov 19, 2014

I would expect Test A and Test B to execute as part of the Suite. That was the one difference between executing via the "Suite" button compared to executing via the "Test" button. I did a check on 20131110, and it's behaving the same (tests in tests are also executed as part of a suite).

@six42

This comment has been minimized.

Contributor

six42 commented Nov 19, 2014

For me the new behaviour sounds pe rfect.
Set the skip property of a test page if you don't want to get it tested as part of a suite.

Am 19. November 2014 10:22:55 MEZ, schrieb Arjan Molenaar notifications@github.com:

I would expect Test A and Test B to execute as part of the Suite. That
was the one difference between executing via the "Suite" button
compared to executing via the "Test" button. I did a check on 20131110,
and it's behaving the same (tests in tests are also executed as part of
a suite).


Reply to this email directly or view it on GitHub:
#538 (comment)

@woodybrood

This comment has been minimized.

Collaborator

woodybrood commented Nov 19, 2014

I'm still curious to know why a Test Page is under a Test Page. What is the
value of this? If this is an accident, then I think seeing the test run
under the Suite would be a reasonable reminder to address it.

I would agree that if you don't want that test to run, the ignore bit is
the thing to do. But When would you want it to run?

Daniel A. Woodward

FitNesse using Slim and Java
@woodybrood
http://whotestedthis.com

On Wed, Nov 19, 2014 at 5:41 AM, six42 notifications@github.com wrote:

For me the new behaviour sounds pe rfect.
Set the skip property of a test page if you don't want to get it tested as
part of a suite.

Am 19. November 2014 10:22:55 MEZ, schrieb Arjan Molenaar <
notifications@github.com>:

I would expect Test A and Test B to execute as part of the Suite. That
was the one difference between executing via the "Suite" button
compared to executing via the "Test" button. I did a check on 20131110,
and it's behaving the same (tests in tests are also executed as part of
a suite).


Reply to this email directly or view it on GitHub:
#538 (comment)


Reply to this email directly or view it on GitHub
#538 (comment).

@ALuckyGuy

This comment has been minimized.

ALuckyGuy commented Nov 23, 2014

First off, I'm fine with it staying as is now, given that's how it worked previously, but I still find the behavior doesn't quite match what I'd expect.

Second, while I don't have much use for test pages under a test page either, there are a few instances I think it makes sense. Since we often create our page hierarchy organizationally logical, and FitNesse is a wiki as well as a test executor, it makes sense to keep related information together, including include pages, old copies of test pages or test data, etc. Having to move old copies of test pages out from under the logical hierarchy you'd already defined is a hassle. If my page tests my widget functionality, doesn't it make sense to store all the info I have related to widgets here under it? The fact that the test hierarchy is actually a true directory hierarchy lends itself to this interpretation too - where else should I store my widget info but in my widget sub-wiki?

Third, the way I see it - and I suspect I'm not alone - in a hierarchy child pages are controlled by their parent. Suite pages have immediate child pages they execute. If those child pages are Suites, grandchild page executions are controlled by their immediate parent suite not the grandparent suite, and Test pages by definition execute themselves and nothing beneath them. So the question is, who controls the children of a Test page? As a leaf node, and regardless of what might be beneath a Test - nothing or many other pages - what's beneath shouldn't execute if the parent page, the Test page in this case, doesn't execute it. There might be includes, old versions of the test page, supporting docs, files whatever, but I see that as controlled by the parent page. If a Test page doesn't execute it's child pages, why should it's parent pages?

FitNesse's implementation of the Skip attribute seems to support this notion too. Setting Skip on either a Suite or Test pages disables the child pages beneath it, which implies that a parent controls their immediate children. Just like a top-level Suite won't execute a grandchild Test page if it's parent Suite doesn't execute it, I don't think a have a top-level suite should directly execute a grandchild Test page if the parent of the Test page doesn't execute it.

So I'm not sure if this is just as logical to others as it seems to me, but to me it makes sense anyway and fits with how Skip works today too.

Also, just for the record, e: creating Test pages under Test pages, I don't think it's the common case and I think a warning would be good idea in any circumstance if there's no !contents command in the hierarchy or link that would display the page you're creating. Pages get lost sometimes, especially with new users. I really wish we could quickly toggle an explorer-like view of the wiki pages too so you could find lost pages without having to go add !contents commands to pages just to see the child pages beneath them. I find new users are frequently creating pages and then losing them because the parent to a new page doesn't have a !contents command.

Finally, @six42 , you said "Set the skip property of a test page if you don't want to get it tested as part of a suite." While I certainly can, that was a solution without rolling back amolenaar's change and would have alleviated the issues you had with it too. If we wanted child pages to execute, we would have defined the parent page as a suite. We didn't want them to execute automatically with the parent though... which is why the parent is defined as Test page instead. IMO the grandparent page shouldn't execute a page automatically either if the parent doesn't.

@six42

This comment has been minimized.

Contributor

six42 commented Nov 28, 2014

Hi David and ALuckyGuy,

look at the "splitting names" example at http concordion.org and let me know how you would buid this in FitNesse without test pages below test pages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment