Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logictestccl: add gateway region as metamorphic test variable #73970

Open
Tracked by #77439
rytaft opened this issue Dec 17, 2021 · 4 comments · May be fixed by #78212
Open
Tracked by #77439

logictestccl: add gateway region as metamorphic test variable #73970

rytaft opened this issue Dec 17, 2021 · 4 comments · May be fixed by #78212
Labels
A-multiregion Related to multi-region A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects

Comments

@rytaft
Copy link
Collaborator

rytaft commented Dec 17, 2021

Metamorphic testing in logictests supports varying many different settings while ensuring that the output of each SQL query is always the same. We should add an extra metamorphic variable for ccllogictests, which varies the region of the gateway node. This may allow us to uncover bugs like #73024, since INSERT behavior is different depending on the region of insertion.

Jira issue: CRDB-11832

@rytaft rytaft added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-testing Testing tools and infrastructure O-postmortem Originated from a Postmortem action item. labels Dec 17, 2021
@rytaft rytaft added this to Triage in SQL Queries via automation Dec 17, 2021
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Dec 17, 2021
@rytaft rytaft changed the title ccl: add gateway region as metamorphic test variable logictestccl: add gateway region as metamorphic test variable Dec 17, 2021
@rytaft rytaft added the A-multiregion Related to multi-region label Dec 17, 2021
@mgartner mgartner moved this from Triage to 22.1 High Likelihood (90%) in SQL Queries Jan 4, 2022
@srosenberg
Copy link
Member

@rytaft This is great! Is the idea simply to randomize query.nodeIdx instead of the current behavior which defaults to 0 unless otherwise specified (in testdata)?

@rytaft
Copy link
Collaborator Author

rytaft commented Mar 18, 2022

@srosenberg That sounds right to me -- I haven't dug into the specifics of how it currently works, but that sounds like the right approach.

@cucaroach cucaroach self-assigned this Mar 21, 2022
@cucaroach cucaroach linked a pull request Apr 6, 2022 that will close this issue
@rytaft rytaft moved this from 22.1 High Likelihood (90%) to 22.2 High Likelihood (90%) in SQL Queries May 13, 2022
@cucaroach
Copy link
Contributor

Two thoughts: reduce noise/intrusiveness by scoping to ccllogic tests, also maybe limit to setting the metamorphic node id on file basis instead of query by query.

@rytaft rytaft moved this from Up Next to Backlog in SQL Queries Oct 27, 2022
@rytaft
Copy link
Collaborator Author

rytaft commented Oct 27, 2022

Moving to backlog since it seems like this is harder than we thought before.

@mgartner mgartner removed the O-postmortem Originated from a Postmortem action item. label Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-multiregion Related to multi-region A-testing Testing tools and infrastructure C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-sql-queries SQL Queries Team
Projects
Status: Backlog
SQL Queries
Backlog (DO NOT ADD NEW ISSUES)
Development

Successfully merging a pull request may close this issue.

4 participants