Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make conn_id unique in Connections table #8608

Closed
ashb opened this issue Apr 28, 2020 · 0 comments · Fixed by #9067
Closed

Make conn_id unique in Connections table #8608

ashb opened this issue Apr 28, 2020 · 0 comments · Fixed by #9067
Labels
area:MetaDB Meta Database related issues. area:providers kind:feature Feature Requests

Comments

@ashb
Copy link
Member

ashb commented Apr 28, 2020

s we discussed in the thread on Spec thread for our new API, the idea
came up of making conn_id unique in Airflow

https://lists.apache.org/thread.html/rfb6f7d95b2754fda9dd09b08444214dfad12d10f143d32de0fcf4104%40%3Cdev.airflow.apache.org%3E

The people in that thread seemed to be in favour of it:

  • It is often confusing to users (there have been a few bug reports about it over the years)
  • It's questionable if it actually works well or not
  • There are better/smarter tools for loadbalancing connections to a DB than picking one of a random list
  • For Hive at least it has been implemented another way - allowing two host's in a single connection string [AIRFLOW-3888] HA for metastore connection #4708 (available in 1.10.6)
  • It makes the HTTP API confusing (needing a integer connection ID, and a string conn_id field)

Given the downsides/work arounds, and the confusion this causes we should remove this (mis)feature from Airflow 2.0.

We should take some care in the migration that does this to capture the unique constraint error and report it a a meaninful way to the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:MetaDB Meta Database related issues. area:providers kind:feature Feature Requests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant