Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecated unused/unmaintained components #311

Merged
merged 12 commits into from
Jul 6, 2023
Merged
5 changes: 4 additions & 1 deletion CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@
[Unreleased](https://github.com/bird-house/birdhouse-deploy/tree/master) (latest)
------------------------------------------------------------------------------------------------------------------

[//]: # (list changes here, using '-' for each new entry, remove this when items are added)
- Deprecate unused/unmaintained components

Move unused and unmaintained components to a separate [`deprecated-components/`](birdhouse/deprecated-components)
subdirectory and remove them from the `DEFAULT_CONF_DIRS` list if required.

[1.26.11](https://github.com/bird-house/birdhouse-deploy/tree/1.26.11) (2023-07-04)
------------------------------------------------------------------------------------------------------------------
Expand Down
7 changes: 7 additions & 0 deletions birdhouse/components/cowbird/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,10 @@ export COWBIRD_MONGODB_PORT=27017

# Subdirectory containing the user workspaces used by Cowbird
export USER_WORKSPACES="user_workspaces"

# this dependency is only required if COWBIRD_MONGODB_SERVICE is the one provided in config/mongodb.
# (include this by default to support backwards compatibility for now)
COMPONENT_DEPENDENCIES="
$COMPONENT_DEPENDENCIES
./config/mongodb
"
8 changes: 8 additions & 0 deletions birdhouse/config/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,10 @@
# Old paths. Keep these so that old config files remain uncommittable after updates.
postgres-magpie/
catalog/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for remembering this to not break autodeploy !

frontend/
malleefowl/
ncops/
ncwms2/
project-api/
solr/
phoenix/

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./config/thredds/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/thredds-providers.cfg:ro
35 changes: 35 additions & 0 deletions birdhouse/config/thredds/config/magpie/providers.cfg.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
providers:
thredds:
url: http://thredds:8080/twitcher/ows/proxy/thredds
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as per other comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not included in #331 since this is being moved from the catalog component.

If you'd like I can make this use the external port 8083 and then change it again when/if #331 gets pulled in but it seems like we might as well do it right the first time (in this PR) since this is new here.

title: Thredds
public: true
c4i: false
type: thredds
sync_type: thredds
# below is a custom config to indicate how magpie should convert thredds path elements into resources/permissions
# see: https://pavics-magpie.readthedocs.io/en/latest/services.html#servicethredds
configuration:
skip_prefix: "thredds" # prefix to ignore, below prefixes will be matched against whatever comes after in path
file_patterns:
# note: make sure to employ quotes and double escapes to avoid parsing YAML error
- ".+\\.ncml" # match longest extension first to avoid tuncating it by match of sorter '.nc'
- ".+\\.nc"
metadata_type:
prefixes:
- null # note: special YAML value evaluated as `no-prefix`, use quotes if literal value is needed
- "\\w+\\.gif" # threddsIcon, folder icon, etc.
- "\\w+\\.ico" # favicon
- "\\w+\\.txt" # licence
- "\\w+\\.css" # tds.css
- "catalog\\.\\w+" # note: special case for `THREDDS` top-level directory (root) accessed for `BROWSE`
- catalog
- ncml
- uddc
- iso
data_type:
prefixes:
- fileServer
- dodsC
- wcs
- wms
- ncss
6 changes: 0 additions & 6 deletions birdhouse/default.env
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ or a custom web interface.
export DEFAULT_CONF_DIRS='
./config/proxy
./config/canarie-api
./config/phoenix
./config/geoserver
./config/flyingpigeon
./config/finch
Expand All @@ -50,9 +49,4 @@ export DEFAULT_CONF_DIRS='
./config/magpie
./config/twitcher
./config/jupyterhub
./config/frontend
./config/project-api
./config/catalog
./config/malleefowl
./config/solr
'
29 changes: 29 additions & 0 deletions birdhouse/deprecated-components/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
Deprecated Components
#####################

.. contents::

All components in this directory are not actively maintained.
mishaschwartz marked this conversation as resolved.
Show resolved Hide resolved

If you wish to include these components in the deployed stack, additional work
may be required to make them compatible with the current stack.

Components in this directory may be removed from this repository at any time.

This directory also contains additional configurations for these deprecated components in:

- `all-public-access-deprecated/`
- `canarie-api-full-monitoring-deprecated/`
- `wps-healthchecks-deprecated/`
Comment on lines +13 to +17
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this again... Is there actually any reason to move them under a separate configuration?

If I activated for example:

EXTRA_CONF_DIRS="
  ./components/weaver
  ./deprecated-components/malleefowl
  ./optional-components/all-public-access-deprecated
  ./optional-components/canarie-api-full-monitoring
  ./optional-components/wps-healthchecks
"

Wouldn't the following be loaded (not necessarily exhaustive or in the right order below)?

./components/weaver/*
./deprecated-components/malleefowl
./optional-components/all-public-access-deprecated/mallefowl/*
./optional-components/all-public-access-deprecated/weaver/*
./optional-components/canarie-api-full-monitoring/mallefowl/*
./optional-components/canarie-api-full-monitoring/weaver/*
./optional-components/wps-healthchecks/mallefowl/*
./optional-components/wps-healthchecks/weaver/*

In order words, whether the main services are under config, components or deprecated-components, the optional-components to use would load whichever service name that it matches nested under its directory.

I find this would be easier to maintain because a service moving between config, components or deprecated-components states would not require updating the optional-components, nor require loading another (optional+deprecated)-component variant equivalent to the original one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't the following be loaded (not necessarily exhaustive or in the right order below)?

At the moment yes but that is a mistake. I just forgot to delete the catalog and malleefowl directories under optional-components/wps-healthchecks/

I've remove them now.

I find this would be easier to maintain because a service moving between config, components or deprecated-components states would not require updating the optional-components, nor require loading another (optional+deprecated)-component variant equivalent to the original one.

I think that you need to think about whether you actually want to deprecate these components or not. In my opinion, the point of moving them into a separate folder is to clearly mark that they are deprecated and no longer maintained. Developers then can focus on maintaining the other components without worrying about breaking components that we don't care about anymore.

The only reason I can think of to keep supporting code in the optional-components folder is if you think that these components will be "un-deprecated" at some point.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone needs to work on, for example, optional-components/wps-healthchecks to add a new WPS service, this doesn't require them to update all nested components. The optional-components/wps-healthchecks/malleefowl for example would be left untouched. I don't think leaving them under optional-components will cause any breaking issue, nor would affect maintenance burden.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you... I moved them back


These contain the settings to extend the deprecated components that have been moved from the corresponding
directories under `birdhouse/optional-components`.

To enable these additional configurations; add them to the `EXTRA_CONF_DIRS` variable (in `env.local`)
as you would to enable any component. For example, to enable the deprecated malleefowl component as well as the
wps-healthchecks for malleefowl. The `EXTRA_CONF_DIRS` variable should contain:

.. code-block:: shell

./deprecated-components/malleefowl
./wps-healthchecks-deprecated
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/catalog/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/catalog_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/catalog/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/catalog-providers.cfg:ro
- ./deprecated-components/catalog/config/magpie/permissions.cfg:${MAGPIE_PERMISSIONS_CONFIG_PATH}/catalog-permissions.cfg:ro
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ services:
ports:
- "8086:80"
volumes:
- ./config/catalog/catalog.cfg:/home/catalog.cfg
- ./deprecated-components/catalog/catalog.cfg:/home/catalog.cfg
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice ! These deprecated should still work !

- wps_outputs:/var/www/html/wps_results
depends_on:
- postgres
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/frontend/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/frontend_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/frontend/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/frontend:ro
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ services:
ports:
- "3000:3000"
env_file:
- ./config/frontend/frontend.env
- ./deprecated-components/frontend/frontend.env
restart: always
logging: *default-logging
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/malleefowl/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/malleefowl_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/malleefowl/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/malleefowl-providers.cfg:ro
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ services:
- "48091:9001"
volumes:
- ${DATA_PERSIST_ROOT}/datasets:/pavics-data
- ./config/malleefowl/custom.cfg:/opt/birdhouse/src/malleefowl/custom.cfg
- ./deprecated-components/malleefowl/custom.cfg:/opt/birdhouse/src/malleefowl/custom.cfg
depends_on:
- postgres
restart: always
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
magpie:
volumes:
- ./deprecated-components/ncwms2/config/magpie/providers.cfg:${MAGPIE_PROVIDERS_CONFIG_PATH}/ncwms2-providers.cfg:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/ncwms2/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/ncwms2:ro
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ services:
- "48080:9001"
volumes:
- ${DATA_PERSIST_ROOT}/datasets:/pavics-data
- ./config/ncwms2/custom.cfg:/opt/birdhouse/custom.cfg
- ./config/ncwms2/server.xml:/opt/birdhouse/eggs/birdhousebuilder.recipe.tomcat-0.2.9-py2.7.egg/birdhousebuilder/recipe/tomcat/server.xml
- ./deprecated-components/ncwms2/custom.cfg:/opt/birdhouse/custom.cfg
- ./deprecated-components/ncwms2/server.xml:/opt/birdhouse/eggs/birdhousebuilder.recipe.tomcat-0.2.9-py2.7.egg/birdhousebuilder/recipe/tomcat/server.xml
restart: always
logging: *default-logging
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/phoenix/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/phoenix_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ services:
- "38443:38443"
- "9001:9001"
volumes:
- ./config/phoenix/custom.cfg:/opt/birdhouse/src/phoenix/custom.cfg
- ./deprecated-components/phoenix/custom.cfg:/opt/birdhouse/src/phoenix/custom.cfg
- ${SSL_CERTIFICATE}:/opt/birdhouse/etc/nginx/cert.pem
links:
- mongodb
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/project-api/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/project-api_canarie_api_monitoring.py:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/project-api/config/proxy/conf.extra-service.d:/etc/nginx/conf.extra-service.d/project-api:ro
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: "3.4"
services:
proxy:
volumes:
- ./deprecated-components/solr/config/canarie-api/canarie_api_monitoring.py:${CANARIE_MONITORING_EXTRA_CONF_DIR}/solr_canarie_api_monitoring.py:ro
16 changes: 10 additions & 6 deletions birdhouse/env.local.example
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,9 @@ export MAGPIE_ADMIN_USERNAME=admin
# For initial bootstrap only, change in the Magpie Web UI after initial boostrap.
export MAGPIE_ADMIN_PASSWORD=qwertyqwerty!
export TWITCHER_PROTECTED_PATH=/twitcher/ows/proxy
export PHOENIX_PASSWORD=phoenix_pass
export PHOENIX_PASSWORD_HASH=sha256:123456789012:1234567890123456789012345678901234567890123456789012345678901234
export TOMCAT_NCWMS_PASSWORD=ncwmspass
export SUPPORT_EMAIL=helpdesk@example.com
export CMIP5_THREDDS_ROOT=birdhouse/CMIP5/CCCMA
export JUPYTERHUB_ADMIN_USERS="{'admin'}" # python set syntax
export CATALOG_USERNAME=admin-catalog
export CATALOG_PASSWORD=qwerty
export CATALOG_THREDDS_SERVICE=thredds
export POSTGRES_PAVICS_USERNAME=postgres-pavics
export POSTGRES_PAVICS_PASSWORD=postgres-qwerty
export POSTGRES_MAGPIE_USERNAME=postgres-magpie
Expand Down Expand Up @@ -486,3 +480,13 @@ export GEOSERVER_ADMIN_PASSWORD=geoserverpass

# Emu WPS service image if that testing component is enabled
#export EMU_IMAGE="tlvu/emu:watchdog"

#############################################################################
# Deprecated vars (for components in the ./deprecated-components directory)
#############################################################################
export TOMCAT_NCWMS_PASSWORD=ncwmspass
export CATALOG_USERNAME=admin-catalog
export CATALOG_PASSWORD=qwerty
export CATALOG_THREDDS_SERVICE=thredds
export PHOENIX_PASSWORD=phoenix_pass
export PHOENIX_PASSWORD_HASH=sha256:123456789012:1234567890123456789012345678901234567890123456789012345678901234
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
config/canarie-api/canarie_api_full_monitoring.py

# Old paths. Keep these so that old config files remain uncommittable after updates.
canarie_api_full_monitoring.py
config/proxy/canarie_api_full_monitoring.py
config/canarie-api/canarie_api_full_monitoring.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Old paths. Keep these so that old config files remain uncommittable after updates.
canarie_api_full_monitoring.py