Tags for tests and suites

[Pazus]

Introduce annotations to tag tests and suites.
Implement filters on execution procedures to include/exclude suites/tests by tags.

We should discuss if "tag" should be assigned to a suite or "suite_type" with predetermined values fits bettet.

[pesse]

If we introduce tags we should probably also give a way to say "run all tests except tag 'x'".
Use-case: "run all tests except tag 'very long-running integration test'"

What do you think?

[jgebal]

Absolutely.
With current implementation we can do negative for all:
Run scema except path
Run path except subpath
That should be a seperate feature request probably

[lwasylow]

How we would like to invoke it? New parameter? That's simplest option probably. As a path? But then we would need to include a keyword to differeniate between real database object etc.
Maybe something like tag:batch

[jgebal]

We currently use:
:path or name how about *tag or some other start character.
I wouldn't want to have separate parameter for it.

[jgebal]

Tags would be just amotjlher way of expressing suitepaths to include.
We should have a separate issue for exclusions like:
-name,-:path,-*tag

[Pazus]

I vote for providing tags as a separate parameter. I think it’s ortogonal to the scope of tests to search and execute. With path we set a scope of tests, with tags we filter it further. So I vote for another overload with tags parameter accepting a single/list of values that are applied on provided path.

[lwasylow]

It's a valid point that address a name uniqueness domain as well.

[jgebal]

I was thonking a bit differently.
Tags and suites are both used for inclusion of tests just lile we use paths and package/procedure names interchangeably today.

Exclusion would come as a separate feature and would be applicable to all three patterns for specifying elements to run.

Exclusuin could be applied by adding - to element that needs to be excluded.

Tags could be identified in input list by special leading character % just like suitepaths are today :
That way I can specify:

• ut.run('%payment , -%slow');
• ut.run(':org.utplsql.stuff, -org.utplsql.stuff.not_this, %utils, %clob');

Maybe we should start by writing potential input param combinations and describe how it should behave before we make final decission on combining it into path parameter or making it separate parameter.

[lwasylow]

It's a great idea but I'm worried about complexity of syntax. New parameter is it extra option indeed but is easier to remember than a special keyword.

[lwasylow]

Happy with a special character I think the common one is hash so I'm thinking

ut.run('#nightlybatch,#customer_services') run all tests with tag nighltybatch and customer_services

ut.run(':core.batch,#nightlybatch') run all tests on suitepath core.batch with tag nightlybatch

This however can lead to ambiguity when package start with hash.

[lwasylow]

Possible options to callout the tags are below.

1.

• ut.run('ut3$user#.test_expectations_cursor ,#contain')
• ut.run('ut3$user#,#cursor')
• ut.run('ut3$user#:utplsql.test_user.expectations,#cursor')

2.

• ut.run(a_path => 'ut3$user#.test_expectations_cursor',a_tags => 'contain')
• ut.run(a_path =>'ut3$user#',a_tags =>'cursor')
• ut.run(a_path =>'ut3$user#:utplsql.test_user.expectations',a_tags =>'cursor')

Option 1

Pros

• No new parameter, easier call

Cons

• Complicated syntax, user require more education
• Possible false positive when we using a schema or package that starts with # and interpretation as tag

Option 2

Pros

• Simpler syntax and clear identification of option which means user with minimum knowledge can run it
• Clear differentiate between tag and other objects

Cons

• Extra parameter , makes a call a bit more complex and require call by name

Given that tags are treated as a way to filter down a execution path to certain area I'm leaning towards the option 2 as better one.

[Pazus]

I also prefer the second one. It’s simpler and clearer for reading and we read code much more then we write it.
Are there any issues with backwards compatibility for existing scripts?

Just an idea: we have an increasing number of parameter combinations for ut.run procedure. Maybe we can creation ut.run_builder function which returns Object Type implementing a builder pattern with all the defaults set? I wonder how modern IDEs provide code completion for nested object type calls.

[jgebal]

After some thinking...
Option 2 is best from my perspective too.

Builder could be an option to go with but I would separate it from this issue.
Change of invocation pattern will require change in every possible consumer. That is a major breaking change.

We could embed the builder in ut_run type directly.
Builder pattern will be however very unnatural to use for most db engineers.