Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for searching boolean fields, including fixes for various bugs. #13614

Merged
merged 11 commits into from May 8, 2024

Conversation

samwho
Copy link
Collaborator

@samwho samwho commented May 3, 2024

Description

This PR introduces tests for searching against boolean fields, which appear to have been quite broken until now. The PR includes fixes for searching against false values in Lucene, and a variety of fixes required to get searching working for the SQL-based datasources.

@samwho samwho added the firestorm Data/Infra/Revenue Team label May 3, 2024
Comment on lines +120 to +131
function fixBooleanFields({ row, table }: { row: Row; table: Table }) {
for (let col of Object.values(table.schema)) {
if (col.type === FieldType.BOOLEAN) {
if (row[col.name] === 1) {
row[col.name] = true
} else if (row[col.name] === 0) {
row[col.name] = false
}
}
}
return row
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQS, MariaDB, and MySQL appear to store boolean values as numeric 1s or 0s, this function parses those back into proper booleans.

Comment on lines +70 to +84
private findRow(expectedRow: any, foundRows: any[]) {
const row = foundRows.find(foundRow => _.isMatch(foundRow, expectedRow))
if (!row) {
const fields = Object.keys(expectedRow)
// To make the error message more readable, we only include the fields
// that are present in the expected row.
const searchedObjects = foundRows.map(row => _.pick(row, fields))
throw new Error(
`Failed to find row: ${JSON.stringify(
expectedRow
)} in ${JSON.stringify(searchedObjects)}`
)
}
return row
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this so that when tests fail because the row searched for does not match any of the rows returned, the message is much nicer. It used to complain about passing undefined to objectContaining, now it looks like this:

CleanShot 2024-05-03 at 17 35 21@2x

@samwho samwho marked this pull request as ready for review May 3, 2024 16:36
@samwho samwho requested a review from a team as a code owner May 3, 2024 16:36
@samwho samwho requested review from mike12345567 and removed request for a team May 3, 2024 16:36
@samwho samwho requested a review from mike12345567 May 8, 2024 14:54
Copy link
Collaborator

@mike12345567 mike12345567 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the isEmpty improvement!

@samwho samwho merged commit 17eece5 into master May 8, 2024
10 checks passed
@samwho samwho deleted the search-tests-boolean branch May 8, 2024 15:02
@github-actions github-actions bot locked and limited conversation to collaborators May 8, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
firestorm Data/Infra/Revenue Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants