Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: query service: escape dollars in collector config generated for pipelines. #4242

Merged
merged 7 commits into from
Dec 19, 2023

Conversation

raj-k-singh
Copy link
Collaborator

Summary

Right now, using pipeline field values containing $ (eg: attributes.$data) breaks collector config.
This happens because the collector attempts to expand tokens starting with $ as env variables and interpolates them to empty string.

The other issue was that pipeline previews were working just fine.
If the preview had surfaced this error, the user wouldn't have had to find out after saving and deploying the pipeline.

This PR includes the following changes

  • Updates collectorsimulator (pipeline preview) to use the same config provider as the actual collector (file provider with env var substitution extension). This will ensure such issues will not go uncaught by the preview.
  • Unfies pipeline config generation logic across collector simulation and actual config recommendations, and escapes $ by replacing it with $$ in that logic.

Related Issues / PR's

Fixes #4180

@github-actions github-actions bot added the bug Something isn't working label Dec 16, 2023
@raj-k-singh raj-k-singh changed the title Fix: qs: escape dollars in collector config generated for pipelines. Fix: query service: escape dollars in collector config generated for pipelines. Dec 16, 2023
@raj-k-singh raj-k-singh marked this pull request as ready for review December 16, 2023 13:45
@raj-k-singh raj-k-singh force-pushed the fix-qs-log-pipelines-escape-dollars branch from f3e563b to f5ab52f Compare December 18, 2023 15:05
@srikanthccv
Copy link
Member

Are there any other special symbols that we need to escape?

@raj-k-singh
Copy link
Collaborator Author

raj-k-singh commented Dec 19, 2023

Are there any other special symbols that we need to escape?

None else, to the best of my current knowledge. I will keep looking and will take up follow up changes for any I discover

@raj-k-singh raj-k-singh merged commit 581bd07 into develop Dec 19, 2023
11 checks passed
@raj-k-singh raj-k-singh deleted the fix-qs-log-pipelines-escape-dollars branch December 19, 2023 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Log Parsing Pipelines] Processor parse from field path containing $ breaks collector config.
3 participants