Log-courier new deploy picking log files from previous install

[psabhar]

Hello
We redeploy our platform daily from scratch to test end to end orchestration and automation. During redeploy of application we dont clear the logs from previous install.
We see that new log-courier installation picks the logs from old installation and hence we see duplicate logs been shipped to elasticsearch and hence we see spike in Kibana. Is there any way we can tel new installation of log-courier to resume the log shipping from state where old install of log-courier was before the redeploy happens?

[driskell]

If the /var/lib/log-courier structure still exists it should continue where it left off and not re-submit any files. If that structure is lost and re-created, Log Courier should begin from the END of the files and not start from scratch (unless the from-beginning flag was set when it was launched - but it defaults to off IIRC)

I can only think that the log files are being copied during re-deploy and then possibly put back. This would cause Log Courier (if it was resuming from existing /var/lib/log-courier structure) to see them as new files that just appeared as their identifiers would be different. If Log Courier is stopped whilst the copying/restoring of the logs happens and the /var/lib/log-courier is deleted, and it is then started, it should start from END of those files so not duplicate (any "new" start with no existing state only sends new logs, never existing - whereas any resume from existing state picks up everything it missed.)