Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updating ingest script to do logs #59

Closed
wants to merge 2 commits into from
Closed

Conversation

acholyn
Copy link
Contributor

@acholyn acholyn commented Jan 9, 2024

closes #58
shell script that runs ingest will write to a log file the datetime and outcome of the ingest function (which should write which files were ingested) and display a completed message.

@tim-band
Copy link
Contributor

If this is to allow Steve Tinney to log from cron I think a much better way for him to do that would be to use systemd-cat -t cron-ingest or something. This will take care of timestamping, log location, log rotation and so on.

echo "ingest complete"

# Run the ingest script and capture the outcome in journal
systemd-cat -t cron-ingest python3 -m ingest.bulk_upload
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this isn't quite what I was meaning. If bulk_upload produces no output then there will be nothing in the log, so no evidence that this script ran at all, since the "Ingest complete" line doesn't go through systemd-cat. If bulk_upload must always produce output then perhaps this isn't a problem.
I was thinking that whoever wanted to run this could prefix by systemd-cat -t cron-ingest and then we wouldn't have to do anything; this is the normal way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah
The bulk-upload does provide some output: it writes the name of the file it's about to index and then when it's finished doing so, does that improve its use?

@acholyn acholyn closed this Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve ingest logging
2 participants