New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Airflow support for S3 compatible storages #9474
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! |
Yes, it works as long as you specify endpoint url. However, it seems to be broken to serving logs in UI for some users on latest release 1.10.10 (works fine in 1.10.9) |
It doesnt work for me even as I continue to provide HOST in the connection and also other relevant parameters. |
We are currently using Ceph RGW as our airflow cluster logging backend. It works perfect. But setting the connection is a little bit tricky U should not set host in the host form field otherwise set host in the |
hi @dispensable , will try this and update the ticket asap! |
hi @dispensable / all, How do I pass the access_key_id and other parameters to connect to that? can someone please help on that? |
Username/Password fields in respective connection |
hi @dispensable/@dimon222, thanks for your help. I have SUCCESSFULLY enabled pushing airflow logs to s3 compatible bucket by following what was mentioned above,
|
Everywhere is HTTP, but what about HTTPS? airflow doesn't support HTTPS S3 endpoints? I've got a very odd error ( ok, how it works.
|
Hi,
Curious to know about the support for S3 compatible storages like DELL ECS, MINIO ETC
Thanks
The text was updated successfully, but these errors were encountered: