Skip to content

enhancement(enriching): add optional wildcard search parameter #23074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jul 9, 2025

Conversation

nzxwang
Copy link
Contributor

@nzxwang nzxwang commented May 19, 2025

Summary

This PR resolves #22920 by enhancing the enrichment functions with support for optional matching against a wildcard as a fallback. It's implementation is a fallback to finding a match against the wildcard in the indexed function to pass the wildcard-matched rows to the follow-up sequential scan which was also updated to include a fallback to matching against the wildcard in row_equals. In the latter, I've refactored the comparison logic to compare_values as I believe its more readable (but I'm happy to be wrong about this). I also tried to update the documentation but I'm happy to correct this as well.

As a side note, I did consider implementing some delimiter parameter which would allow the user to specify delimiter="|" and pass multiple match values per column in the condition field directly but the implementation would be more complicated than simply specifying a wildcard for all fields. Full regex support would be even better but its implementation would be even more complicated.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

How did you test this PR?

cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
RUST_BACKTRACE=1 cargo vdev test enrichment_tables::file::tests:: 
./scripts/check_changelog_fragments.sh

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the "no-changelog" label to this PR.

@github-actions github-actions bot added the domain: external docs Anything related to Vector's external, public documentation label May 19, 2025
@nzxwang nzxwang changed the title Wildcard enhancement(enriching): add optional wildcard search parameter May 19, 2025
@nzxwang nzxwang marked this pull request as ready for review May 21, 2025 16:56
@nzxwang nzxwang requested review from a team as code owners May 21, 2025 16:56
@buraizu buraizu self-assigned this May 21, 2025
@buraizu buraizu removed their assignment May 21, 2025
@nzxwang
Copy link
Contributor Author

nzxwang commented Jun 11, 2025

Hi @buraizu, thank you for your review and approval. By any chance, are you able to contact one of the vector maintainers to review this PR as well?

@nzxwang
Copy link
Contributor Author

nzxwang commented Jun 24, 2025

Hi @pront , sorry to bug you directly. Would you mind reviewing this PR when you have some time?

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionality looks good. Docs needs some improvement. I also left some nit comments (but those are optional).

@pront pront added the meta: awaiting author Pull requests that are awaiting their author. label Jun 25, 2025
@pront pront added the domain: enrichment_tables Anything related to the Vector's enrichment tables label Jun 25, 2025
@github-actions github-actions bot removed the meta: awaiting author Pull requests that are awaiting their author. label Jul 7, 2025
@nzxwang
Copy link
Contributor Author

nzxwang commented Jul 7, 2025

Hi @pront thank you for the review. I appreciate your diligence, but would it be okay to merge this without the FindOptions refactoring? In the next few weeks I plan on making more contributions to vector and I will complete it after having this wildcard search functionality available to us.

Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@pront pront enabled auto-merge July 7, 2025 19:34
auto-merge was automatically disabled July 8, 2025 17:05

Head branch was pushed to by a user without write access

@pront pront enabled auto-merge July 8, 2025 17:46
auto-merge was automatically disabled July 8, 2025 19:14

Head branch was pushed to by a user without write access

@nzxwang
Copy link
Contributor Author

nzxwang commented Jul 8, 2025

Sorry for all the noise. I didn't realize that the website docs get parsed into actual tests. This time, I confirmed that cargo vdev test-vrl ran properly before pushing.

@pront pront enabled auto-merge July 9, 2025 13:26
@pront
Copy link
Member

pront commented Jul 9, 2025

Sorry for all the noise. I didn't realize that the website docs get parsed into actual tests. This time, I confirmed that cargo vdev test-vrl ran properly before pushing.

No problem, thank you for the fixes!

@pront pront added this pull request to the merge queue Jul 9, 2025
Merged via the queue into vectordotdev:master with commit 1102f89 Jul 9, 2025
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: enrichment_tables Anything related to the Vector's enrichment tables domain: external docs Anything related to Vector's external, public documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

find_enrichment_table_records wildcard match
3 participants