New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Remove existing reports #657

Open
atodorov opened this Issue Dec 8, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@atodorov
Copy link
Member

atodorov commented Dec 8, 2018

We are planning to remove the existing reports feature because they are not well designed and the code internally is a huge blob that can't be extended very easily. Kiwi TCMS team would like to reimplement existing functionality that folks find useful, add new types of reports (incl. nicer graphics and UI) and make it possible for the reporting sub-system to be more easily extendable.

Phase out is planned to begin after 1st March 2019!

Until then we are looking for feedback from the community:

  1. Which of the existing reports do you use? What kind of decisions do you make based on them?

  2. What kind of reports/decision you have to make that currently do not exist or you feel like the information is not easy to obtain ?

@Prome88

This comment has been minimized.

Copy link

Prome88 commented Dec 10, 2018

Currently we only rely on:

  • Test plan progress bar
  • Overall report -> Bug list / View all in Issue Tracker
  • Managing team workload -> some kind of real-time metrics to show progress of individual tester.
  • Managing team predictability -> re-implementation of time tracking and adding a metric that would predict the estimated end of test run etc.
@essoftdev

This comment has been minimized.

Copy link

essoftdev commented Dec 19, 2018

Great, because I've to say the reports... are pretty weak in General

** Test Run Report **
It's very useful to send the feedback to the developer but I think it's not very clear, for example Red color to mark the non-passed tests... will be very useful to just skip the passed ones

We started using the APP few weeks back, so I can not give you much more feedback, but soon I'll post more.

Thanks!

atodorov added a commit that referenced this issue Dec 22, 2018

atodorov added a commit that referenced this issue Dec 22, 2018

@okainov

This comment has been minimized.

Copy link
Contributor

okainov commented Jan 8, 2019

Current (okay, maybe not so current, but from Kiwi from April 2018) reports are really weak, not informative and almost useless.

However, it's still better than nothing. I'd suggest that you do not remove it unless you have proper replacement (probably designed from scratch with nicer templates and content)

@atodorov

This comment has been minimized.

Copy link
Member

atodorov commented Jan 8, 2019

However, it's still better than nothing. I'd suggest that you do not remove it unless you have proper replacement (probably designed from scratch with nicer templates and content)

Oh we are removing those. They are a mess internally and we can't keep them around for much longer. There's no question about it. The removal will be gradual, not everything at once!

If there are specific reports that you use please tell us about them, but I'm certain that is not all of them! Whatever people don't care about will go away.

We're interested in knowing which reports and for what kind of purpose/information you are using them so we can redesign those and make the better.

@okainov

This comment has been minimized.

Copy link
Contributor

okainov commented Jan 8, 2019

@atodorov I use only my own reports that we created in our fork =)

@atodorov

This comment has been minimized.

Copy link
Member

atodorov commented Jan 8, 2019

@okainov can you share more details what these reports are ? Screenshots, some description of what kind of info you are looking for, etc. If you can't share internal information that is understandable but if you can let's hear it.

@okainov

This comment has been minimized.

Copy link
Contributor

okainov commented Jan 9, 2019

@atodorov

Main template for our reports is table view, where each cell of the table is a test run (link).

For the "release", it's something like this - each "build" (mind that it's not the same as none of "Product version" nor "Build" but more like some substring of "Build") is tested against different OSes (which we get from Environment variable which you have removed in master already)

Of course, there is also list of bugs collected from all the test runs with bug summary, exposure, status (which is updated by cron every couple of hours by asking Jira and the second our internal bug tracker)

image

Then for each "build" you have more detailed page where the upper part is just the row from the table above, but below you have some pivot table with test results for components included to this "build" (as before, each cell is a link to test run if exist). As I already told you several times in previous issues, we heavily use Environment variables to set up these links between test runs and components included.
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment