Skip to content

feat(docker): add environment values to set log file for worker and beat#40998

Merged
rusackas merged 4 commits into
apache:masterfrom
iercan:celery-log-file-configuration
Jun 17, 2026
Merged

feat(docker): add environment values to set log file for worker and beat#40998
rusackas merged 4 commits into
apache:masterfrom
iercan:celery-log-file-configuration

Conversation

@iercan

@iercan iercan commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

SUMMARY

This PR introduces two new environment variables, WORKER_LOG_FILE and BEAT_LOG_FILE, to configure log file destinations for the worker and beat in the default Docker environment.

Default values remain unchanged to ensure backward compatibility.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@dosubot dosubot Bot added infra:logging Infra setup - logging install:docker Installation - docker container labels Jun 12, 2026
@bito-code-review

bito-code-review Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #43fdec

Actionable Suggestions - 0
Additional Suggestions - 1
  • docker/docker-bootstrap.sh - 1
    • Misleading variable name for stdout sentinel · Line 70-70
      Rename `HYPHEN_SYMBOL` to `LOG_TO_STDOUT` (or similarly descriptive), add a comment explaining that `-` is the Celery convention for logging to stdout, and remove the obsolete `HYPHEN_SYMBOL` definition to avoid dead code.
Review Details
  • Files reviewed - 1 · Commit Range: b72be93..b72be93
    • docker/docker-bootstrap.sh
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

Comment thread docker/docker-bootstrap.sh Outdated
Comment thread docker/docker-bootstrap.sh Outdated
@bito-code-review

bito-code-review Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #1b8357

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: b72be93..b4f489f
    • docker/docker-bootstrap.sh
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@rusackas rusackas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took this for a spin and I think the default actually does change behavior here, despite the "defaults unchanged" note. When WORKER_LOG_FILE/BEAT_LOG_FILE are unset the flag expands to --logfile=-, and Celery doesn't treat - as stdout... it just opens a file literally named - in the cwd (/app). Confirmed against the 5.4.0 we ship:

_detect_handler("-")  -> WatchedFileHandler /app/-
_detect_handler(None) -> StreamHandler -> <stderr>

So a default docker-compose deploy would stop logging to docker logs and quietly write to a file called - instead. Could we only add the flag when the var is actually set? Something like:

${WORKER_LOG_FILE:+--logfile="$WORKER_LOG_FILE"}

That keeps the current stderr behavior when unset, and drops the need for the HYPHEN_SYMBOL dance.

(fwiw I think the codeant bot's word-split warning is a false alarm here, the expansion is already quoted, and docker/.env is operator-controlled anyway.)

@netlify

netlify Bot commented Jun 16, 2026

Copy link
Copy Markdown

Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit 73e32b6
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6a3121477f410700072d4333
😎 Deploy Preview https://deploy-preview-40998--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@iercan

iercan commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @rusackas for the review. I tested your suggestion for setting and unsetting the destination, and it worked as expected. Updated the code accordingly.

@bito-code-review

bito-code-review Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #1ddb61

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: b4f489f..0eb2ab7
    • docker/docker-bootstrap.sh
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@rusackas rusackas requested a review from eschutho June 17, 2026 17:32

@rusackas rusackas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@rusackas rusackas merged commit 5e812c8 into apache:master Jun 17, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra:logging Infra setup - logging install:docker Installation - docker container size/XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants