Supporting Azure/storage and updating env vars for replication files

[Rub21]

What I am changing

• Adding Azure cli support
• Updating REPLICATION_FOLDER path

How I did it

• Installing Azure cli in osm-processor
• Adding Code for download/upload files to azure storage
• Updating values in Values.yaml and templates
• Azure support for planer-dump, full-history and db-backup

How you can test it

Fro testing, we need to set up credentials in .env.cloudprovider from https://github.com/developmentseed/osm-seed/blob/azure/storage/envs/.env.cloudprovider.example
and then run

docker compose -f compose/replication.yml build
docker compose -f compose/replication.yml up

docker compose -f compose/planet-dump.yml build
docker compose -f compose/planet-dump.yml up

docker compose -f compose/full-history.yml build
docker compose -f compose/full-history.yml up

docker compose -f compose/db-backup-restore.yml build
docker compose -f compose/db-backup-restore.yml up

Related Issues

#228

cc. @geohacker @batpad

[geohacker]

@Rub21 this looks great to me, I think we'll test along with a full setup on AKS. I left one note about the osm-processor version change in the Dockerfiles.

[Rub21]

@geohacker @batpad , the changes here are working good:

• Tested on Azure - storage
• Tested in AWS -S3
• Tested in GCP kubernetes - Cloud storage

[Rub21]

Testing contianers locally:

docker compose -f compose/replication.yml build &
docker compose -f compose/planet-dump.yml build &
docker compose -f compose/full-history.yml build &
docker compose -f compose/db-backup-restore.yml build


docker compose -f compose/replication.yml up &
docker compose -f compose/planet-dump.yml up &
docker compose -f compose/full-history.yml up &
docker compose -f compose/db-backup-restore.yml up


docker compose -f compose/replication.yml down &
docker compose -f compose/planet-dump.yml down &
docker compose -f compose/full-history.yml down &
docker compose -f compose/db-backup-restore.yml down