Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logictest: skip distsql_tenant_locality under duress #118769

Merged

Conversation

rickystewart
Copy link
Collaborator

Flaky test: see #118627

Epic: None
Release note: None

@rickystewart rickystewart requested a review from a team as a code owner February 5, 2024 17:42
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@rickystewart rickystewart force-pushed the skip-distsql-tenant-locality-test branch from 81152b7 to 3b5f45f Compare February 5, 2024 17:46
@rickystewart rickystewart requested a review from a team as a code owner February 5, 2024 17:46
Copy link
Collaborator

@michae2 michae2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You beat me to it! Thanks, sorry I didn't do this earlier. :lgtm:

Reviewed 1 of 12 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rickystewart)


pkg/cmd/generate-logictest/templates.go line 76 at r1 (raw file):

	defer sql.TestingOverrideExplainEnvVersion("CockroachDB execbuilder test version")()
	if file == "distsql_tenant_locality" {
		skip.WithIssue(t, 118627)

We've only seen the failures under stress. Would skip.UnderStress be more targeted?

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @michae2)


pkg/cmd/generate-logictest/templates.go line 76 at r1 (raw file):

Previously, michae2 (Michael Erickson) wrote…

We've only seen the failures under stress. Would skip.UnderStress be more targeted?

I second this - I think skipping under stress is the right approach (some context is in #118559 (comment)).

@yuzefovich
Copy link
Member

We should also revert 051191c as part of the skip.

@rickystewart
Copy link
Collaborator Author

How about skipping under duress? That would include stress as well as race.

@yuzefovich
Copy link
Member

Sounds good to me.

@rickystewart rickystewart force-pushed the skip-distsql-tenant-locality-test branch from 3b5f45f to 0b63d1a Compare February 5, 2024 18:01
@rickystewart rickystewart changed the title logictest: skip distsql_tenant_locality logictest: skip distsql_tenant_locality under duress Feb 5, 2024
@michae2
Copy link
Collaborator

michae2 commented Feb 5, 2024

Is there a way we could put the skip information into the distsql_tenant_locality logictest itself, rather than into the template? I know we have skipif, would that work?

Copy link
Collaborator

@michae2 michae2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(That was just an idea, this is also fine with me.) LGTM

@rickystewart
Copy link
Collaborator Author

Is there a way we could put the skip information into the distsql_tenant_locality logictest itself, rather than into the template? I know we have skipif, would that work?

AFAIK logictests have no "skip under X condition" directives or anything like that with functionality corresponding to the skip package. It would be very convenient.

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Thanks!

Reviewed 13 of 13 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @rickystewart)


pkg/sql/opt/exec/execbuilder/testdata/distsql_tenant_locality line 55 at r2 (raw file):

# - TableReader on SQL Instance 3 to scan Span /106/1/3/0
# - TableReader on SQL Instance 1 to scan Span /106/1/5/0.
query T retry

nit: we should remove retry here too (it was added in the previous attempt at fixing the test 85439c3).

Flaky test: see cockroachdb#118627

Epic: None
Release note: None
@rickystewart rickystewart force-pushed the skip-distsql-tenant-locality-test branch from 0b63d1a to f90b91b Compare February 5, 2024 18:15
@rickystewart
Copy link
Collaborator Author

Should be good now.

@rickystewart
Copy link
Collaborator Author

TFTRs!

bors r=yuzefovich,michae2

@craig
Copy link
Contributor

craig bot commented Feb 5, 2024

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Feb 6, 2024

Build succeeded:

@craig craig bot merged commit aee73ac into cockroachdb:master Feb 6, 2024
9 checks passed
michae2 added a commit to michae2/cockroach that referenced this pull request Feb 7, 2024
In cockroachdb#118769 we added a skip under duress for the distsql_tenant_locality
logic test due to cockroachdb#118627.

Now that the `skip` command supports calling `skip.UnderDuress()`, move
the skip for distsql_tenant_locality from the logictest template to the
test script itself.

Informs: cockroachdb#118627

Epic: None

Release note: None
craig bot pushed a commit that referenced this pull request Feb 7, 2024
118820: logictest: add ability to call skip.UnderStress from Test-script r=rafiss,rickystewart a=michae2

**logictest: add ability to call skip.UnderStress from Test-script**

We already had a `skip` command in our fork of Test-script. Flesh it out
a bit with options for `skip.UnderStress` and family. The new syntax
comes in three forms:

```
skip <ISSUE> [args...]
skip ignorelint [args...]
skip under <deadlock/race/stress/stressrace/metamorphic/duress> [ISSUE] [args...]
```

The first form calls `skip.WithIssue`, the second form calls
`skip.IgnoreLint`, and the third form calls `skip.UnderStress` and
family.

Before this change, the only supported form was `skip [ISSUE]` which
called `skip.IgnoreLint`. With this cange, commands in this original
form will instead call `skip.WithIssue`. AFAIK there are no existing
skip commands currently, so this should be a moot point.

Also fix one mistaken use of `skip` which should have been `skipif`.

Informs: #118627

Epic: None

Release note: None

---

**exec: skip distsql_tenant_locality under duress**

In #118769 we added a skip under duress for the distsql_tenant_locality
logic test due to #118627.

Now that the `skip` command supports calling `skip.UnderDuress()`, move
the skip for distsql_tenant_locality from the logictest template to the
test script itself.

Informs: #118627

Epic: None

Release note: None

Co-authored-by: Michael Erickson <michae2@cockroachlabs.com>
wenyihu6 pushed a commit to wenyihu6/cockroach that referenced this pull request Feb 21, 2024
In cockroachdb#118769 we added a skip under duress for the distsql_tenant_locality
logic test due to cockroachdb#118627.

Now that the `skip` command supports calling `skip.UnderDuress()`, move
the skip for distsql_tenant_locality from the logictest template to the
test script itself.

Informs: cockroachdb#118627

Epic: None

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants