Q: Persist (across > 1 tests) value to variable based on JSON response?

[josdotso]

Example algorithm to be expressed in YAML:

• Create an object given an object name.
• Store in "$FOO" (or similar) the UUID given to object per JSON response.
• Do unrelated tests.
• Perform a GET using "$FOO" (the UUID)

Thanks as always!

[josdotso]

Looks like a duplicate of #78. I'll let @cdent decide if this should be closed though.

[cdent]

One of the main issues with this general idea (also discussed on #78) is how to come up with a reasonable scenario to reference back multiple steps. Your idea takes a somewhat different spin in that you want to be able to say: assign this result to a variable.

I continue to worry that that level of flexibility will lead to unreadable tests and readable tests is one of the main goals of gabbi. However, if you have some ideas on how the syntax might work, please stick them on here so we can hash through it. Enough people have asked for it that it is probably worth considering. For some reason I find your idea a bit more appealing. It feels like the syntax could be less fugly if done right.

[samuelfekete]

We need this as well, for an S3-like multi-part upload, which requires the results of multiple previous responses.

I think a nice interface would be to have something like: $HISTORY['Test Name'].$RESPONSE (or $LOCATION etc). This would make it clear where the result is from, and it won't break the existing usage, which would still be the previous request.

This can be implemented by going up the chain of prior.prior.prior... until a test with the given name is found, then using that in the replacers instead of prior.

[cdent]

I continue to be reluctant about this for the reasons stated in #78.

Which is not to say "no, this is never going to happen", just that it's something where we should make sure there are no other options.

The interface suggested by @samuelfekete is certainly better than anything we've thought of before (especially as it is safe in the face of the number of tests in a suite changing). @FND and @jasonamyers, you want to weigh in?

[samuelfekete]

I can think of workarounds, but they are ugly. Like, having an endpoint I can call that acts as a dictionary.

[samuelfekete]

Happy to contribute a with a pull request if this approach is acceptable.

[jasonamyers]

I really like the idea of $HISTORY. That could be very useful for some test cases I've been avoiding. I'm not a fan of the multiple prior approach as I find it to be very unreadable. Storing of the responses by test name is nice. I'm curious how we might need to handle parallel testing. Would we need to account for something like a depends_on so that we can ensure they are run in the right order?

[cdent]

Would we need to account for something like a depends_on so that we can ensure they are run in the right order?

I'm not sure what you mean on this. The tests are already run in strict first to last order per file/suite. Even if you select just one test, it's priors are run.

In a parallel environment some handling is required in the test harness to make sure grouping is done properly (see the warning).

[jasonamyers]

I agree we are fine with parallelism today. Mostly wondering if we might need something like that in the future to guard against missing history. Either way I really like the idea of the $HISTORY.

[cdent]

Happy to contribute a with a pull request if this approach is acceptable.

Yeah, I've talked around and it seems worth pursuing. I suspect it could take a few iterations to make everyone happy, but the initial idea of $HISTORY['test name'] is sound. So, @samuelfekete, go for it. Thank you very much.

[cdent]

#194 got this in a reasonable fashion