Magellan makes it easier for your data scientists, machine learning engineers, and analysts to discover data within your organization by providing a nicer UI on top of the AWS Glue Data Catalog.
Magellan is currently just an alpha preview.
- Lineage
- OIDC authentication (e.g. Okta)
Magellan might be a good fit for you if:
-
You're on AWS
-
You like the deep integration AWS Glue Data Catalog offers with other AWS servies like Athena, and Hive metastore compatable frameworks such as Apache Spark.
-
You'd like to expose AWS Glue's catalog via a nicer UI / allow employees to authenticate via OIDC without needing AWS credentials.
-
You want to get up and running with a minimal amount of infrastructure (just copy Magellan' React UI to S3 + deploy a Lambda APi to talk to Glue). And you don't want to have to touch Hadoop, K8s and/or EKS and deploy a UI, a search service, a metadata ingestion service, and a databasse just to run your data catalog.
Magellan's goals overlap with other open source offerings such as: Amundsen, Apache Atlas and DataHub.
Generally speaking these other options are more feature rich, but require a lot more infrastructure setup compared to Magellan.