Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to add tag to run to display in 'Runs' list in Dashboard UI #2561

Closed
ArieLevs opened this issue Oct 4, 2018 · 12 comments · Fixed by #5164
Closed

Ability to add tag to run to display in 'Runs' list in Dashboard UI #2561

ArieLevs opened this issue Oct 4, 2018 · 12 comments · Fixed by #5164

Comments

@ArieLevs
Copy link

@ArieLevs ArieLevs commented Oct 4, 2018

Current behavior:

Dashboard UI does not show environment

screen shot 2018-10-04 at 9 15 42

Desired behavior:

Hi,
Is there an option to add environment to the dashboard UI?
So for example if we run sanity tests (or any e2e) for the same project on different environments we have no way to know where did the test ran.

for example screen shot above, test num 100 can run on production, text 101 can run on qa etc...
It would be great to see the env straight on dashboard without entering the test and figuring out where it ran.

Versions

cypress: v3.1.0
os: CentOS 7.3.1611
browser: Electron 59.0.3071.115

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Oct 4, 2018

Hey @ArieLevs check out our new Grouping feature. We've found it helpful in our own tests to specify env the tests ran within, so --group staging and --group dev, etc. This will have all the tests displayed in the same run with labels associating each group.

Let me know if this helps with what you're looking for.

@ArieLevs

This comment has been minimized.

Copy link
Author

@ArieLevs ArieLevs commented Oct 7, 2018

Hi @jennifer-shehane, thanks for the update,
I've ran cypress as cypress run --group arieTest --record --key **** (hard coded --group arieTest so all of our CI tests will use this parameter).

I cat see string arieTest only inside a specific test run view (under Specs) just as shown in the image below Grouping-to-label-parallelization

this way it still forces me to enter the specific test and scroll down to specs, is there any change to bring the environment up front similar to message, run time, etc?

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Oct 9, 2018

We haven't had this feature request before, so there hasn't been any previous discussion on this.

The list of runs would need to display any/all groups within each run.

@jennifer-shehane jennifer-shehane self-assigned this Oct 9, 2018
@jennifer-shehane jennifer-shehane modified the milestones: Sprint 6, Sprint 7 Oct 9, 2018
@LiranBri

This comment was marked as off-topic.

Copy link

@LiranBri LiranBri commented Oct 16, 2018

+1

@lirantal

This comment has been minimized.

Copy link

@lirantal lirantal commented Oct 16, 2018

Hi @jennifer-shehane, this feature will be valuable for us as we normally run cypress tests in multiple environments (development CI, alpha, and production) and need to easily distinguish this from the birds-eye view instead of the actual test run.

@trydionel trydionel modified the milestones: Sprint 7, Sprint 8 Oct 22, 2018
@avnersorek

This comment has been minimized.

Copy link

@avnersorek avnersorek commented Dec 24, 2018

Even just simple tagging/labeling of tests runs, that can be passed in to the CLI and displayed in the Dashboard UI, would be great.

@amrwahdan3

This comment has been minimized.

Copy link

@amrwahdan3 amrwahdan3 commented May 13, 2019

i concur with adding tagging of test runs in the birds eye view (/runs). displaying the value here from the --group flag would be awesome.

the Message displays there but in our case it isnt very helpful b/c the Message seems to be the comments from whatever the last commit was to that branch (we are running through azure)

@jennifer-shehane jennifer-shehane removed this from the Sprint 8 milestone May 29, 2019
@jennifer-shehane jennifer-shehane changed the title Adding test environment to dashboard UI Adding test environment / category / label to 'Runs' list in Dashboard UI Jun 21, 2019
@bahmutov

This comment has been minimized.

Copy link
Contributor

@bahmutov bahmutov commented Aug 7, 2019

I think as step 1 that might be enough is to display the group name(s) in the run list. Right now we do not show --group name in the list of runs, one can kind of guess that there are 2 groups in the middle run in the image below, but you cannot be sure.

Screen Shot 2019-08-07 at 3 52 30 PM

Only by opening the run details and collapsing the individual specs one can see the group names

Screen Shot 2019-08-07 at 3 53 42 PM

Proposal:

If the run has 1 group, show the group name in the list of runs. If there are multiple groups, concatenate and show their name, but put a text limit.

Something like this (I have replaced time ago column with group name column)

Screen Shot 2019-08-07 at 3 52 30 PM

This could be done quickly, since we already have this data in the database, plus the Test Runner has the --group flag feature.

@jennifer-shehane

This comment has been minimized.

Copy link
Member

@jennifer-shehane jennifer-shehane commented Sep 9, 2019

Our plan moving forward is to support 'tags' that can be passed into cypress run via a --tag2 flag.

We would then display those tags in the 'Runs' UI of the Dashboard, allowing filtering by tag.

Some examples of how we think this may be used - also examples with --group flag, to distinguish how you may use groups and tags together.

All of these would have the --record flag.

cypress run --tag develop
cypress run --tag staging
cypress run --tag production
cypress run --tag nightly
cypress run --tag hourly
cypress run --group desktop --tag staging
cypress run --group mobile --tag staging

Adds both tags 'staging, prod'

cypress run --group desktop --tag staging
cypress run --group mobile --tag prod
cypress run --group web-es --tag production
cypress run --group web-en --tag production
cypress run --group desktop-uk --tag develop,nightly
cypress run --group desktop-de --tag develop,nightly

Allows spaces within tag

cypress run --group desktop-uk --tag "mobile es"

New Runs list design

14e38fb6-bd28-449f-9361-38b4baa73911

Test Runner tasks:

  • Capture --tag from CLI
  • Capture tags from Module API

Dashboard tasks:

  • Display 'tag' per run
  • Allow filtering by 'run'
  • Display all tags used per project. Allow setting tag 'colors.

Docs tasks:

  • Document --tag flag.
  • Document `tags' in Module API
  • Document why you would want to use tags (where??)
  • Document tags in the Dashboard 'runs'
@ZoroQualityAutomata

This comment has been minimized.

Copy link

@ZoroQualityAutomata ZoroQualityAutomata commented Sep 16, 2019

Document why you would want to use tags (where??)

As for where to document this, perhaps alongside the docs on the --group and --parallel flags, because --tags is another way to organize and group tests together by type.

@jennifer-shehane jennifer-shehane changed the title Adding test environment / category / label to 'Runs' list in Dashboard UI Ability to add tag to run to display in 'Runs' list in Dashboard UI Sep 18, 2019
@jennifer-shehane jennifer-shehane mentioned this issue Sep 19, 2019
3 of 4 tasks complete
@cypress-bot

This comment has been minimized.

Copy link

@cypress-bot cypress-bot bot commented Dec 5, 2019

The code for this is done in cypress-io/cypress#5164, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-bot

This comment has been minimized.

Copy link

@cypress-bot cypress-bot bot commented Dec 12, 2019

Released in 3.8.0.

@cypress-io cypress-io locked as resolved and limited conversation to collaborators Dec 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
9 participants
You can’t perform that action at this time.