Skip to content

Add support for syncing to BigQuery#6

Merged
patrickeasters merged 1 commit intomasterfrom
bigquery
Apr 30, 2026
Merged

Add support for syncing to BigQuery#6
patrickeasters merged 1 commit intomasterfrom
bigquery

Conversation

@patrickeasters
Copy link
Copy Markdown
Member

@patrickeasters patrickeasters commented Apr 30, 2026

CheckMeIn data (visits, unlocks, certifications) currently lives only in a local SQLite database with no path to cross-referencing it with other org data. By syncing to BigQuery, we can query it alongside our Salesforce data and use LLMs to answer operational questions that span both systems — e.g. correlating member visit history with membership status, or identifying certified members who haven't been in the shop recently.

Summary

  • Adds POST /sync/ — a Bearer token-protected endpoint that reads four SQLite tables and uploads them to BigQuery via full table replace (WRITE_TRUNCATE)
  • Supports both service account JSON key file and VM-attached IAM identity (Application Default Credentials)
  • Response includes row counts per table: {"status": "ok", "rows_uploaded": {"visits": N, "guests": N, "unlocks": N, "certifications": N}}

Tables synced

Table Key fields
visits rowid, start, leave, barcode, status
guests guest_id, displayName, email, firstName, lastName, whereFound, status, newsletter
unlocks time, location, barcode
certifications user_id, tool_id, tool_name, certifier_id, date, level

Note: certifications is joined to tools so tool_name is included directly — no separate lookup needed in BigQuery.

Configuration

Add to production.conf:

sync.token           : 'your-secret-token'
bigquery.credentials : ''   # leave empty to use VM IAM identity (ADC)
bigquery.project     : 'your-gcp-project-id'
bigquery.dataset     : 'checkmein'

@patrickeasters patrickeasters merged commit 0c0cd03 into master Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant