Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Application status does not change from "Deploying" on Error #195

Open
rtpro opened this issue Nov 19, 2022 · 2 comments
Open

Application status does not change from "Deploying" on Error #195

rtpro opened this issue Nov 19, 2022 · 2 comments
Assignees
@vadim-zabolotniy
Labels
API / Backend bug Something isn't working

Comments

@rtpro
Copy link
Contributor

rtpro commented Nov 19, 2022

Steps to reproduce:

  1. Deploy the app in the same namespace it already exists
  2. Application status changes to "Deploying"
  3. Get the error below in terminal
  4. app status stays in "Deploying"
application_1    | INFO:     172.21.0.1:60348 - "GET /api/v1/user/me HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60356 - "GET /api/v1/organization/kubernetes-configuration HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60348 - "GET /api/v1/application/list HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60352 - "GET /api/v1/template/list HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60352 - "GET /api/v1/kubernetes/namespace/list?context_name=jovianx-admin-jovianx-system-gke_jovianx-dev-workload_us-central1-a_jovianx-dev HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60352 - "OPTIONS /api/v1/application/install HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:60352 - "POST /api/v1/application/install HTTP/1.1" 200 OK
task-executor_1  | INFO:procrastinate.worker.worker:Starting job application__run_pre_install_hooks[17815](application_id=15)
task-executor_1  | INFO:procrastinate.jobs:Deferred job application__install_components[17816](application_id=15)
task-executor_1  | INFO:procrastinate.worker.worker:Job application__run_pre_install_hooks[17815](application_id=15) ended with status: Success, lasted 0.087 s
task-executor_1  | INFO:procrastinate.worker.worker:Starting job application__install_components[17816](application_id=15)
task-executor_1  | ERROR:managers.applications:Failed to install application component. Failed to install chart "bitnami/redis". Release with name "redis" already exists.
task-executor_1  | ERROR:procrastinate.worker.worker:Job application__install_components[17816](application_id=15) ended with status: Error, lasted 8.525 s
task-executor_1  | Traceback (most recent call last):
task-executor_1  |   File "/home/app/hub/services/helm/subcommands/install.py", line 69, in chart
task-executor_1  |     output = await self._run_command(command)
task-executor_1  |   File "/home/app/hub/services/helm/subcommands/base.py", line 56, in _run_command
task-executor_1  |     return await run(command=command, environment=self.environment, directory=directory)
task-executor_1  |   File "/home/app/hub/utils/shell.py", line 94, in run
task-executor_1  |     raise NonZeroStatusException(
task-executor_1  | exceptions.shell.NonZeroStatusException: Error: INSTALLATION FAILED: cannot re-use a name that is still in use
task-executor_1  | 
task-executor_1  | 
task-executor_1  | During handling of the above exception, another exception occurred:
task-executor_1  | 
task-executor_1  | Traceback (most recent call last):
task-executor_1  |   File "/home/app/hub/managers/applications.py", line 286, in install_component
task-executor_1  |     return await self.helm_manager.install_chart(
task-executor_1  |   File "/home/app/hub/managers/helm/manager.py", line 86, in install_chart
task-executor_1  |     return await helm_service.install.chart(
task-executor_1  |   File "/home/app/hub/services/helm/subcommands/install.py", line 75, in chart
task-executor_1  |     raise ReleaseAlreadyExistsException(
task-executor_1  | exceptions.helm.ReleaseAlreadyExistsException: Failed to install chart "bitnami/redis". Release with name "redis" already exists.
task-executor_1  | 
task-executor_1  | During handling of the above exception, another exception occurred:
task-executor_1  | 
task-executor_1  | Traceback (most recent call last):
task-executor_1  |   File "/usr/local/lib/python3.10/site-packages/procrastinate/worker.py", line 231, in run_job
task-executor_1  |     task_result = await task_result
task-executor_1  |   File "/home/app/hub/services/procrastinate/tasks/application/install_flow.py", line 62, in install_applicatoin_components
task-executor_1  |     await asyncio.gather(*[
task-executor_1  |   File "/home/app/hub/managers/applications.py", line 298, in install_component
task-executor_1  |     raise ApplicationComponentInstallException(
task-executor_1  | exceptions.application.ApplicationComponentInstallException: Failed to install application component.
task-executor_1  | INFO:procrastinate.periodic:Periodic job application__check_health[17817](timestamp=1668857280) deferred for timestamp 1668857280 with id 17817
task-executor_1  | INFO:procrastinate.worker.worker:Starting job application__check_health[17817](timestamp=1668857280)
task-executor_1  | INFO:procrastinate.worker.worker:Job application__check_health[17817](timestamp=1668857280) ended with status: Success, lasted 2.815 s
application_1    | INFO:     172.21.0.1:39814 - "GET /api/v1/user/me HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:40498 - "GET /api/v1/organization/kubernetes-configuration HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:39814 - "GET /api/v1/application/list HTTP/1.1" 200 OK
application_1    | INFO:     172.21.0.1:40488 - "GET /api/v1/template/list HTTP/1.1" 200 OK
task-executor_1  | INFO:procrastinate.periodic:Periodic job application__check_health[17818](timestamp=1668857340) deferred for timestamp 1668857340 with id 17818
task-executor_1  | INFO:procrastinate.worker.worker:Starting job application__check_health[17818](timestamp=1668857340)
task-executor_1  | INFO:procrastinate.worker.worker:Job application__check_health[17818](timestamp=1668857340) ended with status: Success, lasted 2.792 s
task-executor_1  | INFO:procrastinate.periodic:Periodic job application__check_health[17819](timestamp=1668857400) deferred for timestamp 1668857400 with id 17819
task-executor_1  | INFO:procrastinate.worker.worker:Starting job application__check_health[17819](timestamp=1668857400)
task-executor_1  | INFO:procrastinate.worker.worker:Job application__check_health[17819](timestamp=1668857400) ended with status: Success, lasted 2.712 s
@rtpro rtpro added bug Something isn't working API / Backend labels Nov 19, 2022
@vadim-zabolotniy vadim-zabolotniy self-assigned this Dec 8, 2022
@vadim-zabolotniy
Copy link
Collaborator

Hi @rtpro!
I have few questions to you.

  1. what you actually expect when we trying to install component(Helm chart) that already exists in this namespace? Application's status should become error or deployed?
  2. How you determine that application doesn't change its status? I tried to confirm that issue on current main and unable to reproduce it. I checked application in DB shell and in list of applications API endpoint in both of them application have status: error and health: unhealthy.

@rtpro
Copy link
Contributor Author

rtpro commented Dec 9, 2022 •
edited

  1. Error with the helm error message in events

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vadim-zabolotniy vadim-zabolotniy

Labels
API / Backend bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

195-application-status-change-on-error JovianX/Service-Hub
2 participants
@rtpro @vadim-zabolotniy