diff --git a/docs/snowflake-native-apps/facebook-marketing-shared-application.png b/docs/snowflake-native-apps/facebook-marketing-shared-application.png new file mode 100644 index 00000000000000..dffefe023e953e Binary files /dev/null and b/docs/snowflake-native-apps/facebook-marketing-shared-application.png differ diff --git a/docs/snowflake-native-apps/facebook-marketing.md b/docs/snowflake-native-apps/facebook-marketing.md index 7d06510d0b6625..525b62a3d07c42 100644 --- a/docs/snowflake-native-apps/facebook-marketing.md +++ b/docs/snowflake-native-apps/facebook-marketing.md @@ -21,8 +21,11 @@ Do not refresh the Apps page while the application is being installed. This may 2. On the left sidebar, click `Marketplace`. 3. Search for `Facebook Marketing Connector` by Airbyte or navigate to https://app.snowflake.com/marketplace/listing/GZTYZ9BCRT8/airbyte-facebook-marketing-connector-by-airbyte 4. Click `Request`. This will send a request that we will manually service as soon as we can. + +![](./facebook-marketing-shared-application.png) + 5. On the left sidebar, click `Apps`. -6. Under the `Recently Shared with You` section, you should see the `Facebook Marketing Connector by Airbyte`. Click `Get`. +6. Under the `Recently Shared with You` section, you should see the `Airbyte Facebook Marketing Connector` by Airbyte. Click `Get`. 7. Expand `Options`. 1. You can rename the application or leave the default. This is how you will reference the application from a worksheet. 2. Specify the warehouse that the application will be installed to. @@ -37,23 +40,23 @@ In order for the Facebook Marketing Connector by Airbyte to query Facebook's API ## Snowflake Native App Authorizations :::note -By default the app will be installed using the name `AIRBYTE_FACEBOOK_MARKETING`, but if you renamed the app during installation, you will have to use that name as a reference. +By default the app will be installed using the name `AIRBYTE_FACEBOOK_MARKETING_CONNECTOR`, but if you renamed the app during installation, you will have to use that name as a reference. ::: 1. Create the database where the app will access the authorization. ``` -CREATE DATABASE airbyte_facebook_marketing_db; -USE airbyte_facebook_marketing_db; +CREATE DATABASE AIRBYTE_FACEBOOK_MARKETING_DB ; +USE AIRBYTE_FACEBOOK_MARKETING_DB; ``` 2. The native app will validate the output database and create it if it does not exist. In order to do that, the app needs access to the database: ``` -GRANT CREATE DATABASE ON ACCOUNT TO APPLICATION ; +GRANT CREATE DATABASE ON ACCOUNT TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; ``` 3. You will need to allow outgoing network traffic based on the domain of the source. In the case of Facebook Marketing, simply run: ``` -CREATE OR REPLACE NETWORK RULE facebook_marketing_apis_network_rule +CREATE OR REPLACE NETWORK RULE FACEBOOK_MARKETING_APIS_NETWORK_RULE MODE = EGRESS TYPE = HOST_PORT VALUE_LIST = ('graph.facebook.com'); @@ -65,7 +68,7 @@ As of 2023-09-13, the [Snowflake documentation](https://docs.snowflake.com/en/sq 4. Once you have external access configured, you need define your authorization/authentication. Provide the credentials to the app as such: ``` -CREATE OR REPLACE SECRET airbyte_app_secret +CREATE OR REPLACE SECRET AIRBYTE_APP_SECRET TYPE = GENERIC_STRING SECRET_STRING = '{ "access_token": "" @@ -75,26 +78,32 @@ CREATE OR REPLACE SECRET airbyte_app_secret 5. Once the network rule and the secret are defined in Snowflake, you need to make them available to the app by using an external access integration. ``` -CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION airbyte_app_integration +CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION AIRBYTE_APP_INTEGRATION ALLOWED_NETWORK_RULES = (facebook_marketing_apis_network_rule) - ALLOWED_AUTHENTICATION_SECRETS = (airbyte_app_secret) + ALLOWED_AUTHENTICATION_SECRETS = (AIRBYTE_APP_SECRET) ENABLED = true; ``` 6. Grant permission for the app to access the integration. ``` -GRANT USAGE ON INTEGRATION airbyte_app_integration TO APPLICATION AIRBYTE_FACEBOOK_MARKETING; +GRANT USAGE ON INTEGRATION AIRBYTE_APP_INTEGRATION TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; ``` 7. Grant permissions for the app to access the database that houses the secret and read the secret. ``` -GRANT USAGE ON DATABASE airbyte_facebook_marketing_db TO APPLICATION AIRBYTE_FACEBOOK_MARKETING; -GRANT USAGE ON SCHEMA public TO APPLICATION AIRBYTE_FACEBOOK_MARKETING; -GRANT READ ON SECRET airbyte_app_secret TO APPLICATION AIRBYTE_FACEBOOK_MARKETING; +GRANT USAGE ON DATABASE AIRBYTE_FACEBOOK_MARKETING_DB TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; +GRANT USAGE ON SCHEMA PUBLIC TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; +GRANT READ ON SECRET AIRBYTE_APP_SECRET TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; +``` + +8. Grant permissions for the app to create a warehouse on which to execute sync tasks, and to execute tasks. +``` +GRANT CREATE WAREHOUSE ON ACCOUNT TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; +GRANT EXECUTE TASK ON ACCOUNT TO APPLICATION AIRBYTE_FACEBOOK_MARKETING_CONNECTOR; ``` -## Configure a connection +## Configure a Connection Once this is all set up, you can now configure a connection. To do so, use the Streamlit app by going in the `Apps` section and selecting `AIRBYTE_FACEBOOK_MARKETING`. You will have to accept the Anaconda terms in order to use Streamlit. Once you have access to the app, select `New Connection` and fill the following fields: @@ -109,13 +118,13 @@ The Facebook Ad account ID to use when pulling data from the Facebook Marketing `start_date` -UTC date in the format 2021-09-29T12:13:14Z. Any data before this date will not be replicated. +UTC date in the format YYYY-MM-DDTHH:mm:ssZ (e.g. 2021-09-29T12:13:14Z). Any data before this date will not be replicated. --- `end_date` -UTC date in the format 2021-09-29T12:13:14Z. Any data after this date will not be replicated. +UTC date in the format YYYY-MM-DDTHH:mm:ssZ (e.g. 2021-09-29T12:13:14Z). Any data after this date will not be replicated. --- @@ -161,7 +170,7 @@ The table where the schema will be saved. Snowflake's table [naming convention]( --- -`Connection name` +`Connection Name` How the connection will be referred in the Streamlit app. @@ -169,17 +178,36 @@ How the connection will be referred in the Streamlit app. `Replication Frequency` -How often records are fetched. +The sync schedule that determines how often your data will be synced to the target database. --- ## Enabling Logging and Event Sharing for an Application Sharing the logging and telemetry data of your installed application helps us improve the application and can allow us to better triage problems that your run into. To configure your application for logging and telemetry data please refer to the documentation for [Enabling Logging and Event Sharing](event-sharing.md). -## Run a sync -Once a connection is configured, go in `Connections List` and click on `Sync Now` for the connection you want to sync. Once the sync is complete, you should be able to validate that the records have been stored in `.` - -### Supported Streams +## Syncing Your Facebook Marketing Data +Once a connection is configured, go in `Connections List` to view all of your connections. From here for each connection you can +view the configuration settings, start a sync, and view the prior sync history. + +### Scheduled Syncs +While creating a connection, you can specify a "Replication Frequency" which will dictate how often your data will be extracted from +Facebook Marketing and loaded into your Snowflake database. This process is started automatically according to your schedule and does not +require that you manually trigger syncs. For example, if you create a connection at 10:15 AM and set your replication frequency to +hourly, then a sync will be started immediately. The next sync will start at 11:15 AM onwards. Only one sync is active at the same +time. In the event that your sync runs longer than one hour, a new sync will start at the next available time. + +### Manual Syncs +In addition to scheduled syncs, you can also configure a connection to only sync data on-demand by setting "Replication Frequency" to +`MANUAL`. After creating a connection, from the `Connections List` page, you can use the "Sync Now" button to trigger a sync of +your API data to your Snowflake database. You can also use this button to manually trigger connections that sync according to a +schedule. If there is already a sync in progress, this button will be disabled. + +### Sync History +From the `Connections List` page, you can view information about past syncs for each connection to determine when your +data is done syncing and whether the operation was successful. Once the sync is completed successfully, you should be +able to validate that the records have been stored in `.`. + +## Supported Streams As of now, all supported streams perform a full refresh. Incremental syncs are not yet supported. Here are the list of supported streams: * Activities * Ad Account diff --git a/docs/snowflake-native-apps/linkedin-ads-shared-application.png b/docs/snowflake-native-apps/linkedin-ads-shared-application.png new file mode 100644 index 00000000000000..ad4d42f0f489af Binary files /dev/null and b/docs/snowflake-native-apps/linkedin-ads-shared-application.png differ diff --git a/docs/snowflake-native-apps/linkedin-ads.md b/docs/snowflake-native-apps/linkedin-ads.md index 84dcc34384f06f..b029bbf943a0ce 100644 --- a/docs/snowflake-native-apps/linkedin-ads.md +++ b/docs/snowflake-native-apps/linkedin-ads.md @@ -21,8 +21,11 @@ Do not refresh the Apps page while the application is being installed. This may 2. On the left sidebar, click `Marketplace`. 3. Search for `Linkedin Ads Connector` by Airbyte or navigate to https://app.snowflake.com/marketplace/listing/GZTYZ9BCRSJ/airbyte-linkedin-ads-connector-by-airbyte 4. Click `Request`. This will send a request that we will manually service as soon as we can. + +![](./linkedin-ads-shared-application.png) + 5. On the left sidebar, click `Apps`. -6. Under the `Recently Shared with You` section, you should see the `Linkedin Ads Connector by Airbyte`. Click `Get`. +6. Under the `Recently Shared with You` section, you should see the `Airbyte LinkedIn Ads Connector` by Airbyte. Click `Get`. 7. Expand `Options`. 1. You can rename the application or leave the default. This is how you will reference the application from a worksheet. 2. Specify the warehouse that the application will be installed to. @@ -37,23 +40,23 @@ In order for the LinkedIn Ads Connector by Airbyte to query LinkedIn, you will n ## Snowflake Native App Authorizations :::note -By default the app will be installed using the name `AIRBYTE_LINKEDIN_ADS`, but if you renamed the app during installation, you will have to use that name as a reference. +By default the app will be installed using the name `AIRBYTE_LINKEDIN_ADS_CONNECTOR`, but if you renamed the app during installation, you will have to use that name as a reference. ::: 1. Create the database where the app will access the authorization. ``` -CREATE DATABASE airbyte_linkedin_ads_db; -USE airbyte_linkedin_ads_db; +CREATE DATABASE AIRBYTE_LINKEDIN_ADS_DB; +USE AIRBYTE_LINKEDIN_ADS_DB; ``` 2. The native app will validate the output database and create it if it does not exist. In order to do that, the app needs access to the database: ``` -GRANT CREATE DATABASE ON ACCOUNT TO APPLICATION ; +GRANT CREATE DATABASE ON ACCOUNT TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; ``` 3. You will need to allow outgoing network traffic based on the domain of the source. In the case of LinkedIn Ads, simply run: ``` -CREATE OR REPLACE NETWORK RULE linkedin_apis_network_rule +CREATE OR REPLACE NETWORK RULE LINKEDIN_APIS_NETWORK_RULE MODE = EGRESS TYPE = HOST_PORT VALUE_LIST = ('api.linkedin.com', 'www.linkedin.com', 'linkedin.com'); @@ -65,7 +68,7 @@ As of 2023-09-13, the [Snowflake documentation](https://docs.snowflake.com/en/sq 4. Once you have external access configured, you need define your authorization/authentication. Provide the credentials to the app as such: ``` -CREATE OR REPLACE SECRET airbyte_app_secret +CREATE OR REPLACE SECRET AIRBYTE_APP_SECRET TYPE = GENERIC_STRING SECRET_STRING = '{ "auth_method": "oAuth2.0", @@ -78,35 +81,33 @@ CREATE OR REPLACE SECRET airbyte_app_secret 5. Once the network rule and the secret are defined in Snowflake, you need to make them available to the app by using an external access integration. ``` -CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION airbyte_app_integration - ALLOWED_NETWORK_RULES = (linkedin_apis_network_rule) - ALLOWED_AUTHENTICATION_SECRETS = (airbyte_app_secret) +CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION AIRBYTE_APP_INTEGRATION + ALLOWED_NETWORK_RULES = (LINKEDIN_APIS_NETWORK_RULE) + ALLOWED_AUTHENTICATION_SECRETS = (AIRBYTE_APP_SECRET) ENABLED = true; ``` 6. Grant permission for the app to access the integration. ``` -GRANT USAGE ON INTEGRATION airbyte_app_integration TO APPLICATION AIRBYTE_LINKEDIN_ADS; +GRANT USAGE ON INTEGRATION AIRBYTE_APP_INTEGRATION TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; ``` 7. Grant permissions for the app to access the database that houses the secret and read the secret. ``` -GRANT USAGE ON DATABASE airbyte_linkedin_ads_db TO APPLICATION AIRBYTE_LINKEDIN_ADS; -GRANT USAGE ON SCHEMA public TO APPLICATION AIRBYTE_LINKEDIN_ADS; -GRANT READ ON SECRET airbyte_app_secret TO APPLICATION AIRBYTE_LINKEDIN_ADS; +GRANT USAGE ON DATABASE AIRBYTE_LINKEDIN_ADS_DB TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; +GRANT USAGE ON SCHEMA PUBLIC TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; +GRANT READ ON SECRET AIRBYTE_APP_SECRET TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; ``` 8. Grant permissions for the app to create a warehouse on which to execute sync tasks, and to execute tasks. ``` -GRANT CREATE WAREHOUSE ON ACCOUNT TO APPLICATION AIRBYTE_LINKEDIN_ADS; -GRANT EXECUTE TASK ON ACCOUNT TO APPLICATION AIRBYTE_LINKEDIN_ADS; +GRANT CREATE WAREHOUSE ON ACCOUNT TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; +GRANT EXECUTE TASK ON ACCOUNT TO APPLICATION AIRBYTE_LINKEDIN_ADS_CONNECTOR; ``` -## Configure a connection -Once this is all set up, you can now configure a connection. To do so, use the Streamlit app by going in the `Apps` section and selecting `AIRBYTE_LINKEDIN_ADS`. You will have to accept the Anaconda terms in order to use Streamlit. - -![](./linkedin-ads-ui.gif) +## Configuring a Connection +Once this is all set up, you can now configure a connection. To do so, use the Streamlit app by going in the `Apps` section and selecting `AIRBYTE_LINKEDIN_ADS`. You will have to accept the Anaconda terms in order to use Streamlit. Once you have access to the app, select `New Connection` and fill the following fields: @@ -114,7 +115,7 @@ Once you have access to the app, select `New Connection` and fill the following `start_date` -UTC date in the format 2020-09-17. Any data before this date will not be replicated. +UTC date in the format YYYY-MM-DD (e.g. 2020-09-17). Any data before this date will not be replicated. --- @@ -136,7 +137,7 @@ The table where the schema will be saved. Snowflake's table [naming convention]( --- -`Connection name` +`Connection Name` How the connection will be referred in the Streamlit app. @@ -144,17 +145,36 @@ How the connection will be referred in the Streamlit app. `Replication Frequency` -How often records are fetched. +The sync schedule that determines how often your data will be synced to the target database. --- ## Enabling Logging and Event Sharing for an Application Sharing the logging and telemetry data of your installed application helps us improve the application and can allow us to better triage problems that your run into. To configure your application for logging and telemetry data please refer to the documentation for [Enabling Logging and Event Sharing](event-sharing.md). -## Run a sync -Once a connection is configured, go in `Connections List` and click on `Sync Now` for the connection you want to sync. Once the sync is complete, you should be able to validate that the records have been stored in `.` - -### Supported Streams +## Syncing Your LinkedIn Ads Data +Once a connection is configured, go in `Connections List` to view all of your connections. From here for each connection you can +view the configuration settings, start a sync, and view the prior sync history. + +### Scheduled Syncs +While creating a connection, you can specify a "Replication Frequency" which will dictate how often your data will be extracted from +LinkedIn Ads and loaded into your Snowflake database. This process is started automatically according to your schedule and does not +require that you manually trigger syncs. For example, if you create a connection at 10:15 AM and set your replication frequency to +hourly, then a sync will be started immediately. The next sync will start at 11:15 AM onwards. Only one sync is active at the same +time. In the event that your sync runs longer than one hour, a new sync will start at the next available time. + +### Manual Syncs +In addition to scheduled syncs, you can also configure a connection to only sync data on-demand by setting "Replication Frequency" to +`MANUAL`. After creating a connection, from the `Connections List` page, you can use the "Sync Now" button to trigger a sync of +your API data to your Snowflake database. You can also use this button to manually trigger connections that sync according to a +schedule. If there is already a sync in progress, this button will be disabled. + +### Sync History +From the `Connections List` page, you can view information about past syncs for each connection to determine when your +data is done syncing and whether the operation was successful. Once the sync is completed successfully, you should be +able to validate that the records have been stored in `.`. + +## Supported Streams As of now, all supported streams perform a full refresh. Incremental syncs are not yet supported. Here are the list of supported streams: * Accounts * Account Users