Sentry in docker has no notification email

[cuirongqing]

I install sentry in docker. I can see all events but I cannot receive any notification mail.
It's strange that the test mail can notify successfully.

• Sentry 8.0.5
• Python 2.7.11

I have made sure that

• Mail plugin is enabled and project is enabled in http:/localhost:8080/account/settings/notifications
  
• The queue works fine.
  
• I did not see any error in log.
  `{"log":"[2016-02-02 06:06:02,417: INFO/MainProcess] Scheduler: Sending due task send-ping (sentry.tasks.send_ping)\n","stream":"stderr","time":"2016-02-02T06:06:02.418920036Z"}
  {"log":"[2016-02-02 06:06:02,421: INFO/MainProcess] Scheduler: Sending due task check-auth (sentry.tasks.check_auth)\n","stream":"stderr","time":"2016-02-02T06:06:02.421568292Z"}
  {"log":"[2016-02-02 06:06:02,619: INFO/MainProcess] Scheduler: Sending due task schedule-digests (sentry.tasks.digests.schedule_digests)\n","stream":"stderr","time":"2016-02-02T06:06:02.620291103Z"}
  {"log":"[2016-02-02 06:06:04,723: INFO/MainProcess] Scheduler: Sending due task flush-buffers (sentry.tasks.process_buffer.process_pending)\n","stream":"stderr","time":"2016-02-02T06:06:04.724734059Z"}
  {"log":"[2016-02-02 06:06:04,730: INFO/MainProcess] Scheduler: Sending due task sync-options (sentry.tasks.options.sync_options)\n","stream":"stderr","time":"2016-02-02T06:06:04.734650187Z"}
  {"log":"[2016-02-02 06:06:14,732: INFO/MainProcess] Scheduler: Sending due task flush-buffers (sentry.tasks.process_buffer.process_pending)\n","stream":"stderr","time":"2016-02-02T06:06:14.732939296Z"}
  {"log":"[2016-02-02 06:06:14,757: INFO/MainProcess] Scheduler: Sending due task sync-options (sentry.tasks.options.sync_options)\n","stream":"stderr","time":"2016-02-02T06:06:14.760265938Z"}
  {"log":"[2016-02-02 06:06:24,763: INFO/MainProcess] Scheduler: Sending due task flush-buffers (sentry.tasks.process_buffer.process_pending)\n","stream":"stderr","time":"2016-02-02T06:06:24.763518561Z"}
  {"log":"[2016-02-02 06:06:24,779: INFO/MainProcess] Scheduler: Sending due task sync-options (sentry.tasks.options.sync_options)\n","stream":"stderr","time":"2016-02-02T06:06:24.779931587Z"}
  {"log":"[2016-02-02 06:06:32,634: INFO/MainProcess] Scheduler: Sending due task schedule-digests (sentry.tasks.digests.schedule_digests)\n","stream":"stderr","time":"2016-02-02T06:06:32.635682837Z"}
  {"log":"[2016-02-02 06:06:34,780: INFO/MainProcess] Scheduler: Sending due task flush-buffers (sentry.tasks.process_buffer.process_pending)\n","stream":"stderr","time":"2016-02-02T06:06:34.781486883Z"}

Please tell me any idea how to find the problem and fix this issue.

Many thanks.
Rongqing

[mattrobenolt]

Are you using our official image? And can you confirm that both web and workers are running the exact same configs? It sounds like email isn't configured correctly for the workers.

[cuirongqing]

@mattrobenolt
Thank you for yr quick response. I'm new coming to docker & sentry. I think it's official image and I pulled this image by "docker pull sentry " command.
Please tell me how I can check both web and workers are running the exact same configs.

[mattrobenolt]

How are you configuring email now? I assume you're just missing the required environment variables on your worker containers.

If you can share what you're using to run everything, I can help you.

[mattrobenolt]

Also, this is a duplicate of #35

[cuirongqing]

@mattrobenolt

# This file is just Python, with a touch of Django which means
# you can inherit and tweak settings to your hearts content.

# For Docker, the following environment variables are supported:
#  SENTRY_POSTGRES_HOST
#  SENTRY_POSTGRES_PORT
#  SENTRY_DB_NAME
#  SENTRY_DB_USER
#  SENTRY_DB_PASSWORD
#  SENTRY_REDIS_HOST
#  SENTRY_REDIS_PORT
#  SENTRY_REDIS_DB
#  SENTRY_MEMCACHED_HOST
#  SENTRY_MEMCACHED_PORT
#  SENTRY_FILESTORE_DIR
#  SENTRY_SERVER_EMAIL
#  SENTRY_EMAIL_HOST
#  SENTRY_EMAIL_PORT
#  SENTRY_EMAIL_USER
#  SENTRY_EMAIL_PASSWORD
#  SENTRY_EMAIL_USE_TLS
#  SENTRY_MAILGUN_API_KEY
#  SENTRY_SECRET_KEY
from sentry.conf.server import *  # NOQA

import os
import os.path

CONF_ROOT = os.path.dirname(__file__)

postgres = os.environ.get('SENTRY_POSTGRES_HOST') or (os.environ.get('POSTGRES_PORT_5432_TCP_ADDR') and 'postgres')
if postgres:
    DATABASES = {
        'default': {
            'ENGINE': 'sentry.db.postgres',
            'NAME': (
                os.environ.get('SENTRY_DB_NAME')
                or os.environ.get('POSTGRES_ENV_POSTGRES_USER')
                or 'postgres'
            ),
            'USER': (
                os.environ.get('SENTRY_DB_USER')
                or os.environ.get('POSTGRES_ENV_POSTGRES_USER')
                or 'postgres'
            ),
            'PASSWORD': (
                os.environ.get('SENTRY_DB_PASSWORD')
                or os.environ.get('POSTGRES_ENV_POSTGRES_PASSWORD')
                or ''
            ),
            'HOST': postgres,
            'PORT': (
                os.environ.get('SENTRY_POSTGRES_PORT')
                or ''
            ),
            'OPTIONS': {
                'autocommit': True,
            },
        },
    }

# You should not change this setting after your database has been created
# unless you have altered all schemas first
SENTRY_USE_BIG_INTS = True

# If you're expecting any kind of real traffic on Sentry, we highly recommend
# configuring the CACHES and Redis settings

###########
# General #
###########

# Instruct Sentry that this install intends to be run by a single organization
# and thus various UI optimizations should be enabled.
SENTRY_SINGLE_ORGANIZATION = True

#########
# Redis #
#########

# Generic Redis configuration used as defaults for various things including:
# Buffers, Quotas, TSDB

redis = os.environ.get('SENTRY_REDIS_HOST') or (os.environ.get('REDIS_PORT_6379_TCP_ADDR') and 'redis')
if not redis:
    raise Exception('Error: REDIS_PORT_6379_TCP_ADDR (or SENTRY_REDIS_HOST) is undefined, did you forget to `--link` a redis container?')

redis_port = os.environ.get('SENTRY_REDIS_PORT') or '6379'
redis_db = os.environ.get('SENTRY_REDIS_DB') or '0'

SENTRY_REDIS_OPTIONS = {
    'hosts': {
        0: {
            'host': redis,
            'port': redis_port,
            'db': redis_db,
        },
    },
}

#########
# Cache #
#########

# Sentry currently utilizes two separate mechanisms. While CACHES is not a
# requirement, it will optimize several high throughput patterns.

memcached = os.environ.get('SENTRY_MEMCACHED_HOST') or (os.environ.get('MEMCACHED_PORT_11211_TCP_ADDR') and 'memcached')
if memcached:
    memcached_port = (
        os.environ.get('SENTRY_MEMCACHED_PORT')
        or '11211'
    )
    CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': [memcached + ':' + memcached_port],
            'TIMEOUT': 3600,
        }
    }

# A primary cache is required for things such as processing events
SENTRY_CACHE = 'sentry.cache.redis.RedisCache'
SENTRY_CACHE_OPTIONS = SENTRY_REDIS_OPTIONS

#########
# Queue #
#########

# See https://docs.getsentry.com/on-premise/server/queue/ for more
# information on configuring your queue broker and workers. Sentry relies
# on a Python framework called Celery to manage queues.

CELERY_ALWAYS_EAGER = False
BROKER_URL = 'redis://' + redis + ':' + redis_port + '/' + redis_db

###############
# Rate Limits #
###############

# Rate limits apply to notification handlers and are enforced per-project
# automatically.

SENTRY_RATELIMITER = 'sentry.ratelimits.redis.RedisRateLimiter'

##################
# Update Buffers #
##################

# Buffers (combined with queueing) act as an intermediate layer between the
# database and the storage API. They will greatly improve efficiency on large
# numbers of the same events being sent to the API in a short amount of time.
# (read: if you send any kind of real data to Sentry, you should enable buffers)

SENTRY_BUFFER = 'sentry.buffer.redis.RedisBuffer'

##########
# Quotas #
##########

# Quotas allow you to rate limit individual projects or the Sentry install as
# a whole.

SENTRY_QUOTAS = 'sentry.quotas.redis.RedisQuota'

########
# TSDB #
########

# The TSDB is used for building charts as well as making things like per-rate
# alerts possible.

SENTRY_TSDB = 'sentry.tsdb.redis.RedisTSDB'

###########
# Digests #
###########

# The digest backend powers notification summaries.

SENTRY_DIGESTS = 'sentry.digests.backends.redis.RedisBackend'

################
# File storage #
################

# Any Django storage backend is compatible with Sentry. For more solutions see
# the django-storages package: https://django-storages.readthedocs.org/en/latest/

SENTRY_FILESTORE = 'django.core.files.storage.FileSystemStorage'
SENTRY_FILESTORE_OPTIONS = {
    'location': os.environ['SENTRY_FILESTORE_DIR'],
}

##############
# Web Server #
##############

# If you're using a reverse SSL proxy, you should enable the X-Forwarded-Proto
# header and uncomment the following settings

if 'SENTRY_USE_SSL' in os.environ:
    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
    SESSION_COOKIE_SECURE = True

SENTRY_WEB_HOST = '0.0.0.0'
SENTRY_WEB_PORT = 9000
SENTRY_WEB_OPTIONS = {
    #'workers': 3,  # the number of gunicorn workers
    # 'secure_scheme_headers': {'X-FORWARDED-PROTO': 'https'},
}

###############
# Mail Server #
###############

# For more information check Django's documentation:
# https://docs.djangoproject.com/en/1.6/topics/email/

email = os.environ.get('SENTRY_EMAIL_HOST') or (os.environ.get('SMTP_PORT_25_TCP_ADDR') and 'smtp')

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.sina.com'
EMAIL_HOST_PASSWORD = '<OMITTED>'
EMAIL_HOST_USER = '<OMITTED>'
EMAIL_PORT = 587
EMAIL_USE_TLS = 'True'

SERVER_EMAIL = '<OMITTED>'

# The email address to send on behalf of
#EMAIL_BACKEND = 'django.core.mail.backends.dummy.EmailBackend'

# The email address to send on behalf of
# SERVER_EMAIL = os.environ.get('SENTRY_SERVER_EMAIL') or 'root@localhost'

# If you're using mailgun for inbound mail, set your API key and configure a
# route to forward to /api/hooks/mailgun/inbound/
MAILGUN_API_KEY = os.environ.get('SENTRY_MAILGUN_API_KEY') or ''

secret_key = os.environ.get('SENTRY_SECRET_KEY')
if secret_key:
    SECRET_KEY = secret_key

[mattrobenolt]

I'm sorry, but this config isn't very helpful.

How are you building/running the containers?

[cuirongqing]

1. docker run -d --name sentry-redis --restart always redis
2. docker run -d --name sentry-postgres -e POSTGRES_PASSWORD=secret -e POSTGRES_USER=sentry --restart always postgres
3. docker run -it --rm --link sentry-postgres:postgres --link sentry-redis:redis sentry upgrade
4. docker run -d --name my-sentry --link sentry-redis:redis --link sentry-postgres:postgres --restart always -p 8080:9000 sentry
5. docker run -d --name sentry-celery-beat --link sentry-postgres:postgres --link sentry-redis:redis ---restart always sentry celery beat
6. docker run -d --name sentry-celery-worker --link sentry-postgres:postgres --link sentry-redis:redis --restart always sentry celery worker

[mattrobenolt]

Yeah, none of this is configuring email. Are you sure the test email even sent?

There are a couple issues here:

• You pasted a config file here that indicates that you're defining SMTP credentials. But in none of these docker commands are you mounting in your custom config file.
• If the only thing you need to change are credentials, you can skip mounting in the file, and just provide environment variables: docker run -e SENTRY_EMAIL_HOST=... These options are documented here: https://hub.docker.com/_/sentry/

So as I see it and with what you pasted, no email at all should be getting sent since you'll be hitting this path:

docker-sentry/8.0/sentry.conf.py

Line 229 in 2e70636

EMAIL_BACKEND = 'django.core.mail.backends.dummy.EmailBackend'

[cuirongqing]

I'm sure the test email sends successfully. I mount the config file to edit SMTP credential and replace to the container file and restart sentry container.

[cuirongqing]

@mattrobenolt Thanks very much.
I will try to run sentry with docker run -e SENTRY_EMAIL_HOST variable environment
and test again.

[mattrobenolt]

Then your instructions that you said don't match what you did. :(

You need to also get this config file into the celery containers as well. The only thing wrong here is the workers don't have the config they need.

[cuirongqing]

@mattrobenolt Thanks very much.

I copy the config file from sentry to celery worker and beat. After restarting worker and sentry, it can notify email now. O(∩_∩)O~~

[Roberto34343]

I have the same problem with the version 8.8.0
I did the same that cuirongqing but it doesnt work. also beat is not any more in sentry?
celery is deprecated?, I put enviroment variables and links to redis and postgres contairnes with worker and cron, also in website of sentry in Queue sentry.tasks.email.send_email shows changes but in the part of the email test shows this "STARTTLS extension not supported by server"

I am using the smtp of gmail
docker run -d --name my-sentry -e SENTRY_SECRET_KEY='the generate secret key' -e SENTRY_SERVER_EMAIL=r.....k@gmail.com -e SENTRY_EMAIL_HOST=smtp.gmail.com -e SENTRY_EMAIL_PORT=587 -e SENTRY_EMAIL_USER=r.....k@gmail.com -e SENTRY_EMAIL_PASSWORD=xxxxxxx -e SENTRY_EMAIL_USE_TLS=True --link my-redis:redis --link my-postgres:postgres --restart always -p 8080:9000 sentry

The same with the worker and cron.

PD: sorry for my english.

[lenybernard]

I had the same problem, I was only receiving the test emal.
I figured out: I had only set the email environment variable for the sentry container when it had also to be done to the celeri sentry-worker and in the sentry-cron.

Here is the configuration which worked for me:

docker run -d --name my-sentry \
    -p 9000:9000 \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-redis:redis \
    --link sentry-postgres:postgres sentry

docker run -d --name sentry-cron \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-postgres:postgres \
    --link sentry-redis:redis sentry run cron

docker run -d --name sentry-worker-1 \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-postgres:postgres --link sentry-redis:redis sentry run worker

[brunis]

@lenybernard nvm, got it working with your example, thanks!

[cdd111]

1. docker run -d --name sentry-redis --restart always redis
2. docker run -d --name sentry-postgres -e POSTGRES_PASSWORD=secret -e POSTGRES_USER=sentry --restart always postgres
3. docker run -it --rm --link sentry-postgres:postgres --link sentry-redis:redis sentry upgrade
4. docker run -d --name my-sentry --link sentry-redis:redis --link sentry-postgres:postgres --restart always -p 8080:9000 sentry
5. docker run -d --name sentry-celery-beat --link sentry-postgres:postgres --link sentry-redis:redis ---restart always sentry celery beat
6. docker run -d --name sentry-celery-worker --link sentry-postgres:postgres --link sentry-redis:redis --restart always sentry celery worker

run this command??

[brunis]

I had the same problem, I was only receiving the test emal.
I figured out: I had only set the email environment variable for the sentry container when it had also to be done to the celeri sentry-worker and in the sentry-cron.

Here is the configuration which worked for me:

docker run -d --name my-sentry \
    -p 9000:9000 \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-redis:redis \
    --link sentry-postgres:postgres sentry

docker run -d --name sentry-cron \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-postgres:postgres \
    --link sentry-redis:redis sentry run cron

docker run -d --name sentry-worker-1 \
    -e SENTRY_SERVER_EMAIL=sentry@my.domain \
    -e SENTRY_EMAIL_HOST=smtp.mailgun.org \
    -e SENTRY_EMAIL_PORT=587 \
    -e SENTRY_EMAIL_USER=postmaster@my.domain \
    -e SENTRY_EMAIL_PASSWORD=pass \
    -e SENTRY_EMAIL_USE_TLS=true \
    -e SENTRY_SECRET_KEY='some_key' \
    --link sentry-postgres:postgres --link sentry-redis:redis sentry run worker

You should use env variables for this, instead of specifying the same over and over.

[cdd111]

sentry-worker and in the sentry-cron

if I set the .env, It will use the same config in sentry-worker and the sentry-cron?

[brunis]

sentry-worker and in the sentry-cron

if I set the .env, It will use the same config in sentry-worker and the sentry-cron?

not sure whether that global .env file get's picked up on, otherwise use --env-file in the docker run -d cmds

[cdd111]

sentry-worker and in the sentry-cron

if I set the .env, It will use the same config in sentry-worker and the sentry-cron?

not sure whether that global .env file get's picked up on, otherwise use --env-file in the docker run -d cmds
I see that I've set the environment variables both worker and cron，but still can't send the error email

[brunis]

1. docker run -d --name sentry-redis --restart always redis
2. docker run -d --name sentry-postgres -e POSTGRES_PASSWORD=secret -e POSTGRES_USER=sentry --restart always postgres
3. docker run -it --rm --link sentry-postgres:postgres --link sentry-redis:redis sentry upgrade
4. docker run -d --name my-sentry --link sentry-redis:redis --link sentry-postgres:postgres --restart always -p 8080:9000 sentry
5. docker run -d --name sentry-celery-beat --link sentry-postgres:postgres --link sentry-redis:redis ---restart always sentry celery beat
6. docker run -d --name sentry-celery-worker --link sentry-postgres:postgres --link sentry-redis:redis --restart always sentry celery worker

run this command??

And after i run this sentry upgrade line and break my sentry, what do i do now?