Skip to main content

​
Overview

Set up BigQuery routing to stream your first-party event data to your own Google Cloud Platform project in near real-time. This guide provides context for choosing BigQuery as your destination and what to expect after setup.
Prerequisites:
  • Active Google Cloud Platform project
  • Permissions to grant IAM roles at the project level
  • BigQuery API enabled (default for new projects)
  • Understanding of your data residency requirements (EU vs US)

​
When to Choose BigQuery

Best for:
  • Publishers already using Google Cloud Platform
  • Teams requiring near real-time data access (approximately 5-minute latency)
  • Organizations with SQL-based analytics workflows
  • Publishers needing automatic schema management
Consider alternatives if:
  • Your primary cloud provider is AWS (consider S3 Streaming)
  • You already have a Snowflake data warehouse

​
Setup Steps

BigQuery Routing enables self-service setup through the Permutive Dashboard.
You do not need to manually create a dataset for the Routing destination. This is handled automatically by our system during the setup process. The only manual step required is granting permissions to our service account.
1

Prepare Your GCP Project

Ensure you have a Google Cloud Platform project with BigQuery enabled. You’ll need:
  • Your GCP Project ID
  • A unique name for the dataset Permutive will create
  • Your preferred data location (US or EU)
2

Navigate to Routing in Permutive Dashboard

Log in to your Permutive Dashboard and navigate to the Routing section from the main navigation menu.
3

Add BigQuery Integration

Click Add Integration and select BigQuery from the list of available destinations.
4

Configure Your BigQuery Destination

Enter the following configuration details:
SettingDescription
ProjectYour GCP project ID
DatasetA unique name for the dataset (must not already exist)
LocationUS or EU based on your data residency requirements
5

Grant Access to Permutive Service Account

The Dashboard will display a Permutive service account email address with the format:
routing-customer-*@permutive-routing-production.iam.gserviceaccount.com
Grant this service account the BigQuery User role at the project level in your GCP IAM settings.
If you manually created the dataset beforehand, grant the BigQuery Data Owner role instead.
6

Confirm Access in Permutive Dashboard

Return to the Permutive Dashboard and click Confirm account access granted.
Only click this button after granting permissions in GCP. If access is incorrect, the setup will fail and you’ll need to restart.
7

Setup Completion

The status will transition to Running when setup is complete.If the status shows Failed, contact Technical Services for assistance.

​
Common Pitfalls

Permission Level Mistake: The most common setup error is granting BigQuery User role at the dataset level instead of the PROJECT level. Always grant at the project level unless you manually created the dataset.
Organization Policies: If you encounter “domain restriction” errors, your GCP organization may block external service accounts. Work with your GCP administrator to add Permutive’s domain (@permutive-routing-production.iam.gserviceaccount.com) to your organization’s allowlist.
Historical Data: Routing is point-in-time only. When you enable routing, only new events going forward will be exported. Historical data is not backfilled automatically. Contact Support if you need historical data.

​
What Happens After Setup

Once routing is active:
  1. Tables are created automatically for each event type (e.g., Pageview_events, VideoView_events)
  2. Events stream in near real-time with approximately 5-minute latency
  3. Schemas update automatically when you add new event types or properties
  4. Daily partitions organize data by event date for efficient querying

​
Understanding Your Data Structure

After setup, you’ll have:
  • Event tables: One table per event type with daily partitions
  • Identities table: identities containing user identity mappings with daily partitions
  • Segment metadata: segment_metadata view showing segment definitions
See the BigQuery integration documentation for detailed schema information.

​
Next Steps

BigQuery Integration

View full integration documentation

Back to Routing

Return to Routing overview