Skip to content
This repository was archived by the owner on May 6, 2020. It is now read-only.
This repository was archived by the owner on May 6, 2020. It is now read-only.

Controller crashes if a deployment is in progress when it's started #903

@arschles

Description

@arschles

Repro steps:

  1. Start a Deis cluster with the DEIS_KUBERNETES_DEPLOYMENTS env var set to "true" in the controller RC manifest (see https://deis.com/docs/workflow/applications/deploying-apps/#tuning-application-settings for details on turning on deployments)
  2. Create a new app (deis create --no-remote pulltest)
  3. deis pull the app (deis pull deis/example-go -a pulltest)
  4. Scale the app to something high, so the deployment takes a long time (deis scale cmd=1000 -a pulltest)
  5. Delete the controller RC (kubectl delete rc deis-controller --namespace=deis)
  6. Create the controller RC again (kubectl create -f $(helmc home)/workspace/charts/workflow-v2.2.0/manifests/deis-controller-rc.yaml)
  7. See logs of the controller pod after it's in the RUNNING state (kubectl logs -f <controller pod>)

After step 7, you should see a traceback that looks like:

Deploying available applications
Traceback (most recent call last):
  File "/app/api/models/app.py", line 528, in deploy
    raise AlreadyExists('Deployment for {} is already in progress'.format(name))
api.exceptions.AlreadyExists: Deployment for pulltest-cmd is already in progress

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/manage.py", line 12, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/__init__.py", line 345, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.5/dist-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/app/api/management/commands/load_db_state_to_k8s.py", line 39, in handle
    application.deploy(rel)
  File "/app/api/models/app.py", line 548, in deploy
    raise ServiceUnavailable(err) from e
api.exceptions.ServiceUnavailable: pulltest-cmd (app::deploy): Deployment for pulltest-cmd is already in progress

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions