New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FileHiveMetastore example/documentation? #11943
Comments
Since you're using AWS, Presto can be configured to use AWS Glue as a Hive Metastore. This is specific to using EMR but the general idea is the same, see here. You can launch an EMR Presto with the checkbox enabled to use Glue as the Hive metastore to see how this ends up being configured on the EC2 instances it uses. If you're not using EMR it should essentially just be configuring Presto's
and using an EC2 instance profile that can read from AWS Glue. Also, if you use AWS Athena (managed Presto) you can issue |
Using Glue/Athena is actually an interesting idea, and I'll likely try that, thanks! I think it could still be valuable though to have some form of examples for using the FileHiveMetaStore, so I'd like to keep this request active/open. |
@ankon AFAIK, |
I'm trying to use presto to query data in ORC files produced by pinterest/secor, stored in S3. My environment doesn't have any Hadoop/Hive setup, rather we use Kafka directly. Using presto against Kafka isn't workable (too slow, and would require infinite retention of the data inside Kafka). My table descriptions are produced automatically based on application data, and I want my users to only interact with presto.
I got things to work with Hive 3.1.1, but the requirements for running a Hive metastore are "scary" (no knowledge otherwise, so this adds quite some operational complexity).
I saw in the source code that there is a FileHiveMetastore which seems like it would remove the need to configure and run an actual Hive metastore server, and it should be easy enough to convert my application knowledge into suitable schemas.
What I would need for that is basically some form of example of how to describe a table for this meta-store, or ideally some pointers to documentation.
My table definition right now looks like this:
The text was updated successfully, but these errors were encountered: