-
Notifications
You must be signed in to change notification settings - Fork 394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Log clean up. #32
Log clean up. #32
Conversation
log-cleanup/airflow-log-cleanup.py
Outdated
if CHILD_PROCESS_LOG_DIRECTORY is not ' ': | ||
DIRECTORIES_TO_DELETE.append( CHILD_PROCESS_LOG_DIRECTORY ) | ||
except Exception: | ||
logging.info("CHILD process directory path not available!!") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use logging.error() and print exception stacktrace
log-cleanup/airflow-log-cleanup.py
Outdated
BASE_LOG_FOLDER='""" + BASE_LOG_FOLDER + """' | ||
BASE_LOG_FOLDER=""{{params.directory}}"" | ||
TYPES=""{{params.type}}"" | ||
DELETE_TYPE=""{{params.deleteType}}"" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is never set based off the task definitions
log-cleanup/airflow-log-cleanup.py
Outdated
@@ -58,17 +73,22 @@ | |||
echo "BASE_LOG_FOLDER: '${BASE_LOG_FOLDER}'" | |||
echo "MAX_LOG_AGE_IN_DAYS: '${MAX_LOG_AGE_IN_DAYS}'" | |||
echo "ENABLE_DELETE: '${ENABLE_DELETE}'" | |||
echo "" | |||
echo "TYPE: '${TYPE}'" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add an extra space so ${TYPE} lines up with the print statements above it
log-cleanup/airflow-log-cleanup.py
Outdated
ENABLE_DELETE_CHILD_LOG = Variable.get("enable_delete_child_log", "False") | ||
logging.info("ENABLE_DELETE_CHILD_LOG " + ENABLE_DELETE_CHILD_LOG) | ||
|
||
if ENABLE_DELETE_CHILD_LOG == "True": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See if you can ignore case because people might enter in all lower case
Added options for delete empty folders and cleanup CHILD_PROCESS_LOG directory