Google Analytics Extractor
A CLI tool for extracting Google Analytics data using Google Reporting API. Can be also used to transform data to various formats suitable for migration to other analytics platforms.
Also see - Goodbye, Google Analytics - Why and How You Should Leave The Platform for more context.
You will need Google Cloud API access for run the CLI:
Navigate to Cloud Resource Manager and click Create Project
- alternatively create project with
gcloud projects create $PROJECT_ID
- alternatively create project with
Navigate to Reporting API and click Enable
Go to credentials page
Click Create credentials, select Service account
Give it a name and make note of service account email. Click Create and Continue
Open Service account page
Select previously created service account, Open Keys tab
Click Add Key and Create New Key. Choose JSON format and download it. (store this securely)
Give SA permissions to GA - guide
- email: SA email from earlier
- role: Viewer
Alternatively see following setup.
To install and run:
pip install ga-extractor ga-extractor --help
ga-extractor --help # Usage: ga-extractor [OPTIONS] COMMAND [ARGS]... # ... # Create config file: ga-extractor setup \ --sa-key-path="analytics-api-24102021-4edf0b7270c0.json" \ --table-id="123456789" \ --metrics="ga:sessions" \ --dimensions="ga:browser" \ --start-date="2022-03-15" \ --end-date="2022-03-19" cat ~/.config/ga-extractor/config.yaml # Optionally, check config ga-extractor auth # Test authentication # Successfully authenticated with user: ... ga-extractor setup --help # For options and flags
- Value for
--table-idcan be found in GA web console - Click on Admin section, View Settings and see View ID field
- All configurations and generated extracts/reports are stored in
- You can also use metrics and dimensions presets using
BASIC, if you're not sure which data to extract
ga-extractor extract # Report written to /home/some-user/.config/ga-extractor/report.json
extract perform raw extraction of dimensions and metrics using the provided configs
You can directly extract and transform data to various formats. Available options are:
- JSON (Default option; Default API output)
- SQL (compatible with Umami Analytics PostgreSQL backend)
ga-extractor migrate --format=CSV # Report written to /home/user/.config/ga-extractor/02c2db1a-1ff0-47af-bad3-9c8bc51c1d13_extract.csv head /home/user/.config/ga-extractor/02c2db1a-1ff0-47af-bad3-9c8bc51c1d13_extract.csv # path,browser,os,device,screen,language,country,referral_path,count,date # /,Chrome,Android,mobile,1370x1370,zh-cn,China,(direct),1,2022-03-18 # /,Chrome,Android,mobile,340x620,en-gb,United Kingdom,t.co/,1,2022-03-18 ga-extractor migrate --format=UMAMI # Report written to /home/user/.config/ga-extractor/cee9e1d0-3b87-4052-a295-1b7224c5ba78_extract.sql # IMPORTANT: Verify the data and check test database before inserting into production instance # To insert into DB (This should be run against clean database): cat cee9e1d0-3b87-4052-a295-1b7224c5ba78_extract.sql | psql -Upostgres -a some-db
You can verify the data is correct in Umami web console and GA web console:
Note: Some data in GA and Umami web console might be little off, because GA displays many metrics based on sessions (e.g. Sessions by device), but data is extracted/migrated based on page views. You can however confirm that percentage breakdown of browser or OS usage does match.
- Poetry (+ virtual environment)
poetry install python -m ga_extractor --help
poetry install ga-extractor --help # Usage: ga-extractor [OPTIONS] COMMAND [ARGS]... # ...