Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add circus + gunicorn option when using run.sh #11772

Merged
merged 12 commits into from
Apr 12, 2021

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Apr 1, 2021

What did you do?

  • Dropped paster support
  • Added circusd to run.sh, can now start new stack with APP_WEBSERVER=dev ./run.sh
  • You can setup a custom circus config file with CIRCUS_CONFIG_FILE=<path_to_circus_config>
  • Added basic --daemon, restart, stop functionality

Lot's of work left to do still.

  • Adjust run_tool_shed.sh
    • May need a celery cron job for building the index, or better yet, celery task so we don't need the cronjob
  • Adjust run_reports.sh
  • Add a non-dev circus config file, maybe as multiple .sample files and copy on first start ?
  • Figure out what to do for setups that use mule job handling but are not starting uwsgi. Error out ?
  • Update tool shed bootstrapping scripts
  • Setup simple file-based logging with --daemon

Why did you make this change?

Implements #10885

How to test the changes?

(select the most appropriate option; if the latter, provide steps for testing below)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

For UI Components

  • I've included a screenshot of the changes

@mvdbeek
Copy link
Member Author

mvdbeek commented Apr 2, 2021

If we want to make this the default we would also need to deprecate Python 3.6 or conditionally run uwsgi if python is < 3.7. Since we haven't really used circus/celery/fastAPI in production yet this might come too early, and we probably only want to add extra effort for our deployers once (upgrade Python, figure out how they want to run Galaxy, maybe switch to installed Galaxy that @natefoo is working on, figure out mule-less job scheduling if they had been using mules).

Nothing stops us from setting up test and main to use circus/celery/fastAPI on test and main without this, so I'm just going to add an additional option so devs/users/deployers can try the new stack easily.

If this is too short gunicorn will restart the worker before they
finish booting.
We know this is running from source, so this seems like the most
explicit way of finding relative config files if they're not specified.
Also fixes a bug if GALAXY_CONFIG_FILE is set to no value.
@natefoo
Copy link
Member

natefoo commented Apr 2, 2021

So, the standard version of Python 3 on EL8 is 3.6. If the new stack is only supported on 3.7+ that's kind of a bummer, but we can make do. I have a miniconda Ansible role that makes getting a newer Python installed pretty trivial for production deployments.

@jmchilton
Copy link
Member

Regardless of Python 3.6 vs 3.7 issues - I generally like the idea of adding functionality and removing old defaults in separate PRs. So I prefer this approach of just working on the new dev stack here. Also it takes the pressure off of removing paste for tool shed and such.

@mvdbeek mvdbeek marked this pull request as ready for review April 7, 2021 12:41
@mvdbeek mvdbeek changed the title [WIP] Make circus + gunicorn default, drop paste support Make circus + gunicorn default, drop paste support Apr 7, 2021
@mvdbeek mvdbeek changed the title Make circus + gunicorn default, drop paste support Add circus + gunicorn option when using run.sh Apr 7, 2021
@mvdbeek mvdbeek added this to the 21.05 milestone Apr 7, 2021
@jmchilton jmchilton merged commit c313635 into galaxyproject:dev Apr 12, 2021
@github-actions
Copy link

This PR was merged without a "kind/" label, please correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants