Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve user typing workaround #9780

Merged

Conversation

SergioAtSUSE
Copy link
Member

Fail if workaround is not successful.
Set typing speed by caller.

Comment on lines 42 to 47
wait_screen_change {
if (defined $args{max_interval}) {
type_string($args{username}, max_interval => $args{max_interval});
} else {
type_string($args{username});
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can do something like this

Suggested change
wait_screen_change {
if (defined $args{max_interval}) {
type_string($args{username}, max_interval => $args{max_interval});
} else {
type_string($args{username});
}
$args{max_interval} //= undef;
...
wait_screen_change {
type_string($args{username}, max_interval => $args{max_interval});

Since the definition of type_string in testapi sets it to 250 if it's not defined

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to be explicit. I prefer to not send undef. If there is no parameter to send, better send no parameter at all.
But, if it is an issue for you I can change it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matter of taste :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to the @foursixnine suggestion

if ($retry == 0) {
$self->enter_userinfo();
} else {
$self->enter_userinfo(max_interval => utils::VERY_SLOW_TYPING_SPEED);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go for something like this, but the inner me says: ]Explicit is better than implicit](https://www.python.org/dev/peps/pep-0020/)

Suggested change
$self->enter_userinfo(max_interval => utils::VERY_SLOW_TYPING_SPEED);
$self->enter_userinfo(max_interval => ($retry)? utils::VERY_SLOW_TYPING_SPEED : undef );

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. there is no gain in doing a one-liner.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey, this is perl, meaning … probably we would need to follow the opposite ;P

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than one way to do it. I choose python way.

@SergioAtSUSE SergioAtSUSE force-pushed the improve_user_typing_workaround branch from 1b4ac70 to 28b8aaa Compare March 13, 2020 15:54
Fail if workaround is not successful.
Set typing speed by caller.
@SergioAtSUSE SergioAtSUSE force-pushed the improve_user_typing_workaround branch from 28b8aaa to 8806de7 Compare March 13, 2020 16:10
@foursixnine foursixnine merged commit d2243b2 into os-autoinst:master Mar 13, 2020
@SergioAtSUSE SergioAtSUSE deleted the improve_user_typing_workaround branch March 13, 2020 16:37
@SeroSun
Copy link
Member

SeroSun commented Mar 17, 2020

@SergioAtSUSE Our tests start fails after this PR merged, any idea?
https://openqa.suse.de/tests/3998220#step/user_settings/11
https://openqa.nue.suse.com/tests/3998970#step/user_settings/11

Is this a side effect of this PR?

@jlausuch
Copy link
Contributor

@SergioAtSUSE Our tests start fails after this PR merged, any idea?
https://openqa.suse.de/tests/3998220#step/user_settings/11
https://openqa.nue.suse.com/tests/3998970#step/user_settings/11

Is this a side effect of this PR?

Also affecting x86 jobs: https://openqa.nue.suse.com/tests/3998970#step/user_settings/11

@SergioAtSUSE
Copy link
Member Author

@SergioAtSUSE Our tests start fails after this PR merged, any idea?
https://openqa.suse.de/tests/3998220#step/user_settings/11
https://openqa.nue.suse.com/tests/3998970#step/user_settings/11

Is this a side effect of this PR?

They didn't start failing, they were failing but shown as orange.
We want failed tests as red.

The last good was failing: https://openqa.suse.de/tests/3982502#step/user_settings/10 but shown orange. Then any dependent image can fail later.

Please try increasing the retries or decreasing the max_interval to make the workaround to work.

@SergioAtSUSE
Copy link
Member Author

@SergioAtSUSE Our tests start fails after this PR merged, any idea?
https://openqa.suse.de/tests/3998220#step/user_settings/11
https://openqa.nue.suse.com/tests/3998970#step/user_settings/11
Is this a side effect of this PR?

Also affecting x86 jobs: https://openqa.nue.suse.com/tests/3998970#step/user_settings/11

yes, x86_64 is also hit by typing issues. But, this PR is not making the workaround to not work, it is making it to not ignore a mistype. If there is a mistype after applying the workaround, then the workaround didn't work and the test shall fail.

@SeroSun
Copy link
Member

SeroSun commented Mar 17, 2020

This is really a good idea. When people find their job task turns into red, they probably manually retrigger those task again and again to green, then all orange turns into green. Fantastic ;)

@SeroSun
Copy link
Member

SeroSun commented Mar 17, 2020

no offense, I really don't think remove orange is a good idea. If people what to care softfail, just mandatory review those softfail.
Maybe it's not the right place to complain this. Thank you for your information. I'll keep an eye on those random fails and make retrigger.

@SergioAtSUSE
Copy link
Member Author

This is really a good idea. When people find their job task turns into red, they probably manually retrigger those task again and again to green, then all orange turns into green. Fantastic ;)

  • Retrigger jobs=wasting machine time
  • Reviewing failures caused by typing issues in any other later module=wasting human time
  • Wasting machine time vs. wasting human time
  • Human is more expensive than machines
  • Reviewer get demotivated by wasting time in those typing issues
  • If it is sporadic it is ok to retrigger until it is green,
  • If it isn't sporadic the workaround has to be improved or the product bug has to be fixed.

Now, because there are more teams annoyed by failing test that they don't care than a team "functional" that cares, I will find a way of only fail on functional job group.

@SergioAtSUSE
Copy link
Member Author

no offense, I really don't think remove orange is a good idea. If people what to care softfail, just mandatory review those softfail.
Maybe it's not the right place to complain this. Thank you for your information. I'll keep an eye on those random fails and make retrigger.

Our job group had more than 20 soft-fails in module user_settings: https://openqa.suse.de/tests/overview?arch=&machine=&modules=user_settings&modules_result=softfailed&distri=sle&version=15-SP2&build=154.1&groupid=110#

Some of then were soft-fail with working workaround, some of then where soft-fail with not working workaround. That means that to "resolve the typing issue" you would need to look into all those and find those which are mistyping after the workaround.

Now, we difference between the working workaround (soft-fail) and not working workaround (fail).

But, as I have already said, I agree with your pain and I am looking for a way to make that tests outside functional ignore again the mistype ;)

@SeroSun
Copy link
Member

SeroSun commented Mar 17, 2020

@SergioAtSUSE thank you very much for your help. I understand your problem now. It's really hard to balance these situations.
I'm with you in the comment of "human is more expensive than machine", so I'm tired to manually retrigger for a random fail issue into suddenly green, because this case is the root one and block all following works. (you can feel by this link https://openqa.nue.suse.com/tests/3998994#dependencies). But if no better choice for most people, I could willing to do in this way.

asmorodskyi added a commit to asmorodskyi/os-autoinst-distri-opensuse that referenced this pull request Mar 17, 2020
After merge of os-autoinst#9780 we start fail test every time there is typo in username
This looks irrelevant for HPC because we not using full username anywhere. So
we okay if there is typo in username
asmorodskyi added a commit to asmorodskyi/os-autoinst-distri-opensuse that referenced this pull request Mar 17, 2020
After merge of os-autoinst#9780 we start fail test every time there is typo in username
This looks irrelevant for HPC because we not using full username anywhere. So
we okay if there is typo in username
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants