feat: added journey and step annotations

[shahzad31]

• fix Enable journey.skip to skip tests #404
  Support syntax like

Following annotation have been added

journey.only will only run this journey
journey.skip will skip this journey
step.skip
step.soft failure of this step will not abort rest of the journey
step.only

-f --force if journey is skipped , then while pushing it will error out, but can be ignore with this flag

  step.skip('launch application again', async () => {
    expect(page.textContent('body')).toContain('Google');
  });

example result

[paulb-elastic]

@shahzad31 what's the use case for skipping a step like this (as opposed to deleting it, commenting it out)? Is it to keep the structure as it was before a change (so step n is always step n, even if step n-1 is removed)? Do you envisage this being a temporary thing a user would do, or permanent?

[shahzad31]

@shahzad31 what's the use case for skipping a step like this (as opposed to deleting it, commenting it out)? Is it to keep the structure as it was before a change (so step n is always step n, even if step n-1 is removed)? Do you envisage this being a temporary thing a user would do, or permanent?

@paulb-elastic it's the most familiar way to in testing frameworks to provide an option like this instead of commenting out , it's especially useful when testing journeys locally. It's a very familiar syntax for especially jest developers.

[vigneshshanmugam]

Left some comments.

• What do you think about adding support for journey.skip which is being asked here - Enable journey.skip to skip tests #404. We need to handle the skipped journeys from being pushed in project based monitor workflow
• Need tests for the skipped steps

[vigneshshanmugam]

I would reuse the -y, we already use this for ignoring some of the users changes on local and forcefully pushing the project monitors. Lets not introduce two things to do the same thing.

[vigneshshanmugam]

Thinking through this again. This would end up costing user and doing harm than good. Lets just error in case of skipped journey and figure out a solution if there is a need.

[shahzad31]

i have remove the -f

[vigneshshanmugam]

Pretty much duplicated. Both of the methods .skip and .only can be invoked by having a common function that does the same thing.

func addAnnotation(name: 'only' | 'skip' , args) { 
    const j = new Journey(...args);
    j[name]= true;
    runner.addJourney(j);
}

journey.only = addAnnotation('only')
journey.skip = addAnnotation('skip')

[shahzad31]

done

[vigneshshanmugam]

Been thinking about this today, I am not convinced that this makes sense. Feels like we are trying to work around the expect.soft problem. Lets re-explore this in a separate PR. Mainly suggesting because once we have the expect.soft support this becomes useless.

If the devs want to complete a step without failing, It would be more ideal to either comment out then failing method expect.blah or change that call to a soft assertion. I dont think anyone would have a step just for a assertion and would call .soft on that whole step instead.

[shahzad31]

i think both are separate use case. This just means user doesn't want a step to skip the rest of journey.
step.soft and expect.soft can co-exist.

[vigneshshanmugam]

Thanks Shahzad, I am really not convinced on the soft usecase for step. Would you suggest some usecase?

A step can error only if there are any expect statements, otherwise it cant unless its a JS bug which users can figure out if they run locally

if users want to soft assert, they can do expect.soft or so if we cannot add support via the PW expect. Lets do it as followup call, unless there is a need for the step.soft. We would be in a better position to add it later than to deprecate it based on what I said above.

[vigneshshanmugam]

Had a look today, There are some typings issue and reporter issues. Will push a fix later today.

[vigneshshanmugam]

@shahzad31 I pushed a fix that deals with few things

• Multiple .only on journey/steps work properly
• Reporter had issues and was displaying displayed journeys where our counts are based on steps. Ignored journeys are never reported to keep it simpler.
• Fixed types where it was marked as optional.
• Step level only journeys are run even if they are failling. Last error will be added to the journey level error.

[vigneshshanmugam]

LGTM