Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not delete output file of File Sink Output on every container/service restart #3687

Closed
canob opened this issue Nov 21, 2023 · 1 comment · Fixed by #3722
Closed

Not delete output file of File Sink Output on every container/service restart #3687

canob opened this issue Nov 21, 2023 · 1 comment · Fixed by #3722
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@canob
Copy link

canob commented Nov 21, 2023

Every time that I restart container/service of Data Prepper, the output file configured in my File Sink Output is deleted.

I think that this is a big problem if you are collecting that results file with other program to send it to other platform, because sometimes a file collector do mistakes when the file that is collecting is deleted and a new one with the same name takes its place.

When Data Prepper is working, appends every new line to the file. Is it possible that after a restart of the container/service of Data Prepper, Data Prepper continue appending new lines to the file after restarting, without deleting it first?

Maybe you can add a config option to File Sink Output to decide that.

@kkondaka kkondaka added enhancement New feature or request and removed untriaged labels Nov 21, 2023
@kkondaka
Copy link
Collaborator

Sounds like a good idea to support an option to append to the file instead of truncating the file if it already exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants