Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move database extension creation into DAG task #76

Closed
gnn opened this issue Dec 17, 2020 · 3 comments
Closed

Move database extension creation into DAG task #76

gnn opened this issue Dec 17, 2020 · 3 comments
Assignees
Labels
🚀 feature New feature or feature request

Comments

@gnn
Copy link
Collaborator

gnn commented Dec 17, 2020

Replacing the current extension creation code with

CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS postgis;

and running it at the start of the pipeline via something like

psql -h localhost -p 54321 -U egon -d egon-data \
  -f src/egon/data/airflow/entrypoints/create_postgis_extension.sql \
  --set ON_ERROR_STOP=True

makes sure that the pipeline always fails early if the extension can't be created. This should prevent #74, whose discussion has a comment with more details, from reoccurring for others.

@gnn gnn added the 🚀 feature New feature or feature request label Dec 17, 2020
@gnn gnn self-assigned this Dec 17, 2020
gnn added a commit that referenced this issue Jan 10, 2021
Since the migrations currently also contain one that creates the
database extensions we need, this also fixes #76 and the Docker entry
point is removed with this commit.
@IlkaCu
Copy link
Member

IlkaCu commented Mar 12, 2021

I faced the problem that extension postgis_raster wasn't created automatically.

@ClaraBuettner
Copy link
Contributor

@gnn I noticed that this will be solved with PR which also includes other things and is still marked as draft. But because we frequently run into this error, do you think that it could make sens to solve this preliminary differently? I could also implement this.

@gnn
Copy link
Collaborator Author

gnn commented Mar 15, 2021

I'm currently solving this, because when supporting multiple databases, initializing the database has to be done differently anyway. Will push a solution today.

@gnn gnn closed this as completed in bfa5be7 Mar 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🚀 feature New feature or feature request
Projects
None yet
Development

No branches or pull requests

3 participants