Migration to Google Cloud

[piojanu]

Hi!

We use Google Cloud and I want to migrate this pipeline to GCP services. Starting with Snowflake->BigQuery, where do I need to start and what to look out for?

Next, I guess, I need to follow this Metaflow on GCP deploy guide, right?

Thanks for your work and this project 🙏

Best,
Piotr

[jacopotagliabue]

Hi @piojanu ! Thanks for reaching out.

Moving from SF to bigQuery should be very easy (make sure to change the dbt adapter accordingly and possibly adjust the queries if some SQL detail is different), but moving Metaflow requires a bit more thinking as AWS Batch is not supported by GCP.

However, now Metaflow can also run on GCP: I'm not an expert there, but you can totally join the Slack to get support, tips and suggestions on that: I'm sure the Outerbounds folks would love to help!