Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify CLI interface #189

Open
cezio opened this issue Feb 15, 2018 · 7 comments · May be fixed by #310
Open

Unify CLI interface #189

cezio opened this issue Feb 15, 2018 · 7 comments · May be fixed by #310
Assignees
Labels
Milestone

Comments

@cezio
Copy link
Contributor

@cezio cezio commented Feb 15, 2018

At the moment, GHC is managed with several endpoints: pavement.py, manage.py, model.py, app.py. This could be normalized, some commands could be moved either to pavement or to manage.py.

@justb4 justb4 added the enhancement label Mar 3, 2018
@justb4

This comment has been minimized.

Copy link
Member

@justb4 justb4 commented Mar 3, 2018

True. Things have grown historically. Though there are several categories of commands, broadly:

  • one-time system setup
  • database management (manage.py, models.py)
  • running GHC: healthchecks (models.py) or HTTP server (app.py)
  • maintain system: like updating translations

Have to think what the best strategy is. We could move everything to pavement.py. That seems the most flexible though may be overwhelming. Or do you recommend other ways, maybe even another CLI tool like http://click.pocoo.org/?

Not for this issue: Would things help if we move to Django at some point? Looks like lots GHC stuff is explicitly programmed (forms, DB upgrades, possibly other commands) that is automated or web-based in Django.

@hannesaddec

This comment has been minimized.

Copy link
Collaborator

@hannesaddec hannesaddec commented Mar 5, 2018

+1 to move everything in one file... with the possibilities to have includes

@cezio

This comment has been minimized.

Copy link
Contributor Author

@cezio cezio commented Mar 5, 2018

Have to think what the best strategy is. We could move everything to pavement.py. That seems the most flexible though may be overwhelming. Or do you recommend other ways, maybe even another CLI tool like http://click.pocoo.org/?

I don't think using paver is a good idea. It's purpose is a bit different, and while it can be used to manipulate the app, it's not suited well to build composed apps. Flask offers cli application facilities, uses click under the hood, so I guess this would be more proper direction.

Not for this issue: Would things help if we move to Django at some point? Looks like lots GHC stuff is explicitly programmed (forms, DB upgrades, possibly other commands) that is automated or web-based in Django.

In general yes, although it's better to do it in early stage. Django takes away a lot of project infrastructure decisions from developers, offering solid help there (migrations ootb, custom commands infrastructure, separate test configuration, auth and user management, pluggable apps etc) , but requires keeping to specific project architecture.

@justb4 justb4 modified the milestones: Version 0.4.0, Version 0.5.0 Nov 5, 2018
@justb4 justb4 modified the milestones: Version 0.5.0, Version 0.6.0 Nov 15, 2018
@justb4 justb4 modified the milestones: Version 0.6.0, Version 0.7.0 Jun 11, 2019
@justb4 justb4 modified the milestones: Version 0.7.0, Version 0.8.0 Oct 27, 2019
@borrob

This comment has been minimized.

Copy link
Collaborator

@borrob borrob commented Dec 9, 2019

I browsed through pavement.py and the documentation and I came to this list of CLI commands:

Paver

  • setup: setup plugin dependencies
  • create_secret_key
  • create: db and superuser account
  • create_hash
  • upgrade: upgrade db
  • refresh_docs: sphinx build
  • clean: clean environment
  • extract_translations
  • add_language_catalogue
  • compile translations
  • update_translations
  • runner_deamon
  • run_healthchecks
  • run_tests

From docs

(And not mentioned with paver):

  • python GeoHealthCheck/models.py create: init database
  • python GeoHealthCheck/app.py: start webapp (optional host and port)
  • python GeoHealthCheck/scheduler.py &: start runner in background
  • python GeoHealthCheck/models.py drop: drop database
  • python GeoHealthCheck/models.py load <datafile.json> [y/n]: load data
  • SQL for resetting user password
  • build documentation with sphinx (has already paver command)

So most of the commands are already accessible with paver and my instinct would be to add/modify the commands in the docs that aren't covered yet. Or would it be better to drop paver altogether and move to click as suggested (or something else) ?

Some of the paver commands use sh and I assume that gives issues on windows machines. How important is that?

IMHO: moving to Django would be an entire different discussion with more sides to it than CLI.

@justb4

This comment has been minimized.

Copy link
Member

@justb4 justb4 commented Dec 10, 2019

@borrob thanks for looking into this. Bit hard for me to estimate. For new projects I (others) usually start with Click. This is the only project I use Paver. I am no expert in this, but purposes of these two seem overlapping, with Paver more geared at the setup/install phase and some development like translations. Also Paver shields from OS/Filesystem-dependencies, so in theory sh() should work also on Win, though not completely. (Maybe even Paver could be used within Click?).

We don't have a real (pip) installation yet for GHC as a Python package. A typical Python package installation/setup would usually install a (click) CLI in a global or venv bin location. See for example pygeoapi.
From there a typical scenario is to install a server instance via that CLI. See for example mapproxy where the CLI mapproxy-util is installed.

In the longer term I would like to see a GHC install from PyPi and instance(s) with this scenario, all commands with options/flags as to run in automatic installs like Docker. Each subcommand handles a set of related actions:

pip install geohealthcheck
# installs GHC CLI e.g. `geohc` (`ghc` is already for Haskell..)
geohc create-instance . # creates server instance, possibly interactively

geohc manage-db create
geohc manage-db init
geohc manage-db upgrade
geohc manage-db export
geohc manage-db import

geohc client ... # commands to talk to (future) API, e.g. bulk import Resources.

geohc serve ...

But this will be a longer path, we need project-discussion for. In the short term is it an idea to keep Paver for the development and install (e.g. it downloads .zip for JS libs) phase and introduce Click for daily/admin management? As part of the Paver install a Click CLI could be installed where we could put all DB/'runserver'-related actions. Just thinking out loud. Replacing all by Click is quite disrupting, there's quite some tricky stuff in pavement.py...

As for Django: yes, seen in many Flask-projects, one starts building/adding components to finally get to what Django has built-in or provides in Django-app-extensions: migrations, (registration-) forms, (DB-)views, REST, translations, encrypted passwords, multiple configs etc. My ideal architecture for the future would be a Django-REST app with a Vue.js (or any other) frontend that only interacts via the GHC-API (no Jinja templating). Maybe, if we leave the DB structure as-is and wrapping all "business/healthcheck" logic, we could develop that as a side project, also a PSC issue.

@borrob

This comment has been minimized.

Copy link
Collaborator

@borrob borrob commented Dec 10, 2019

To summarise the discussion and actions:

Short term solution

  • Keep paver and all current paver commands (in order to not brake any existing tool chains)
  • Introduce click for all "non-install" related tasks
  • copy/shortcut/link as many paver commands under click

Long term solution

  • discuss with PSC on migration of Flask to Django framework
  • discuss with PSC on creating a PyPi (and/or conda) package
  • discuss with PSC on CLI tools and interface
  • discuss with PSC on frontend: jinja vs Vue/Angular/...

Actions

I can start with the short term items. Where do we document the PSC items? New issues? A wiki page?

@justb4

This comment has been minimized.

Copy link
Member

@justb4 justb4 commented Dec 11, 2019

@borrob Good summary! Agree you go ahead with Short Term Solution.

GHC does not have a PSC yet, was also a trigger to start one...Also needed for OSGeo.org incubation AFAIK. We can model after pygeoapi: https://pygeoapi.io/community/psc/. Will put forward on GHC Gitter. @tomkralidis : or have I missed something?

@borrob borrob self-assigned this Dec 12, 2019
This was referenced Jan 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.