Documentation Index
Fetch the complete documentation index at: https://docs.permutive.com/llms.txt
Use this file to discover all available pages before exploring further.
Guides
Issues
FAQ
Overview
Sources is the area within the Connectivity suite where you manage your external data platform connections and imports. From Sources, you can establish connections to your data warehouses and data lakes, configure imports from your chosen tables, and bring that data into Permutive for audience building and activation.Beta ProgramWe’re still in the process of fine-tuning and building out this product, so we’re grateful for your participation in our Beta phase. We aim to provide a smooth experience, however, you may encounter some minor hiccups or unexpected behavior as we continue to refine and improve the product. Your feedback is invaluable during this time, so please don’t hesitate to share your thoughts and experiences with us.
Why Use Sources?
Centralize your data connections — Sources provide a single point of management for all your external data platforms. Rather than managing multiple integration methods, you configure your connections once and reuse them across multiple imports. Enable rich audience building — By connecting your data warehouses and storage systems, you unlock the ability to build cohorts based on data that lives outside of Permutive’s behavioral collection. This includes CRM data, purchase history, offline conversions, and any other user data you store externally. Maintain security and control — Source connections use secure authentication methods and allow you to control exactly which data Permutive can access. You choose which tables and columns to import, keeping sensitive data protected. Support flexible import types — A single source connection can power multiple types of imports: user profile data for trait-based targeting, activity data for time-bound events, user identity data for Identity Graph enrichment, and group identity data for household cohorts.Concepts
Definitions
- Source: An external data platform (such as a data warehouse or cloud storage) that Permutive connects to for importing data. Sources are configured through the Catalog and managed on the Connections page.
- Catalog: The interface within the Connectivity Suite where you browse available source platforms and initiate new connections. The Catalog displays all supported sources with their categories and connection status.
- Connection: A configured link between Permutive and a source platform. Connections store authentication credentials and settings needed to access your data. A single platform can have multiple connections (e.g., connections to different BigQuery projects).
- Import: A configured data pipeline that pulls specific data from a source connection into Permutive. Multiple imports can be created from a single connection.
Supported Sources
Permutive supports connections to the following source platforms:| Platform | Category | Description |
|---|---|---|
| Google BigQuery | Data Warehouse | Connect to BigQuery datasets to import tables containing user data |
| Snowflake | Data Warehouse | Connect to Snowflake databases and schemas for data imports |
| Amazon S3 | Cloud Storage | Connect to S3 buckets containing data files |
| Google Cloud Storage | Cloud Storage | Connect to GCS buckets containing data files |
We’re working towards making more source connection types available. If there is a specific data source you’d like to connect, please contact your Customer Success Manager.
Workflows
Browsing the Catalog
The Catalog page is your starting point for discovering and connecting to new source platforms. Use the Catalog to:- Browse available source connectors
- Search and filter platforms by name or category
You can create multiple connections to a single platform. Even if you already have a connection to a platform (e.g., BigQuery), you can create additional connections to other BigQuery projects or datasets.
Creating a Source Connection
To create a new source connection:- Navigate to the Catalog page within Connectivity
- Find and click on the platform you want to connect to
- Click the Connect button
- Provide a descriptive name for your connection
- Enter the required authentication credentials for the platform
- Configure any additional connection settings
- Click Save to create the connection
Managing Connections
The Connections page provides a centralized view of all your source and destination connections. From here you can:- View connection status (Active, Inactive, Processing, Deprecated)
- See which platform and type each connection belongs to
- Access connection details and settings
| Column | Description |
|---|---|
| Name | The name you provided when creating the connection |
| Source | The platform name (e.g., BigQuery, Snowflake) |
| Type | The connection type (currently “Source” for imports) |
| Created | Date the connection was created |
| Status | Current availability: Active, Inactive, Processing, or Deprecated |
Connection Statuses
Source connections have one of four statuses:- Active: The connection is available and can be used to create imports
- Inactive: The connection has been disabled and cannot be used for imports
- Processing: A new connection is being validated. This typically completes within a few minutes
- Deprecated: The connection is no longer supported or has been retired
Import Types
Once a source connection is active, you can create imports to bring different types of data into Permutive:- User Profile Data — Import static user attributes such as demographics, subscription tiers, or CRM segments for trait-based cohort building and targeting.
- User Activity Data — Import time-stamped event or behavioral data such as purchase history or content interactions for time-bound audience building.
- Identity Graph Data — Import user identity mappings and household graphs to enrich Permutive’s Identity Graph with identifiers and group relationships from your data warehouse. See Importing User Identity and Importing User Group Memberships for step-by-step guides.
Guides
Step-by-step instructions for working with Sources.Connecting to BigQuery
Set up a connection to Google BigQuery
Connecting to Snowflake
Set up a connection to Snowflake
Connecting to Amazon S3
Set up a connection to Amazon S3
Connecting to Google Cloud Storage
Set up a connection to Google Cloud Storage
Creating an Import
Create and configure a data import from an active connection
Actioning Schema Updates (Beta)
Add new columns to an existing import when your source schema changes
Troubleshooting
The following issues may occur when working with Sources:Connection stuck in Processing status
Connection stuck in Processing status
If your connection remains in “Processing” status for more than 15 minutes:
- Verify that the authentication credentials are correct
- Check that the source platform is accessible and not experiencing outages
- Ensure any required network permissions (IP allowlisting, firewall rules) are configured
Connection shows as Inactive unexpectedly
Connection shows as Inactive unexpectedly
A connection may become inactive if:
- Authentication credentials have expired or been revoked
- The source platform configuration has changed
Cannot see tables or data in source
Cannot see tables or data in source
If you can connect but cannot see expected tables:
- Verify the service account or user has read permissions on the specific tables
- Check that you’re connecting to the correct project, dataset, or schema
- Confirm the tables contain data and are not empty
Authentication failed error
Authentication failed error
Authentication failures typically occur due to:
- Incorrect credentials (wrong password, expired token, invalid key file)
- Missing or incorrect project/account identifiers
- Service account not enabled or activated
Platform not available in Catalog
Platform not available in Catalog
If you don’t see your desired platform in the Catalog:
- The connector may not yet be supported
- The connector may be in limited availability or beta
- Your workspace may not have access to certain connectors
Environment Compatibility
Core Product
Sources functionality is available in the Permutive Dashboard:| Functionality | Web Dashboard | API |
|---|---|---|
| Browse Catalog | ||
| Create connections | ||
| Manage connections | ||
| View connection status |
Source management is currently available only through the Permutive Dashboard. API support for programmatic connection management is planned for future releases.
Supported Source Platforms
| Platform | Availability | Notes |
|---|---|---|
| Google BigQuery | Generally Available | Full support for standard and partitioned tables |
| Snowflake | Generally Available | Supports all Snowflake cloud providers (AWS, Azure, GCP) |
| Amazon S3 | Generally Available | Supports CSV and Parquet files |
| Google Cloud Storage | Generally Available | Direct GCS bucket access |
Dependencies
Sources rely on the following products and features being configured for your organization:| Dependency | Required | Description |
|---|---|---|
| Connectivity Suite | ✓ | Sources are part of the Connectivity Suite. Access to Connectivity must be enabled for your workspace. |
| Source Platform Access | ✓ | You must have appropriate permissions and credentials in your source platform to establish connections. |
| Imports | ~ | While not required for creating connections, Imports are needed to actually bring data from Sources into Permutive. |
Limits
Sources adhere to the following product specifications and limits.Feature Limits
| Feature | Description | Limit |
|---|---|---|
| Connections per platform | Number of connections you can create to a single platform type | No limit |
| Connections per workspace | Total number of source connections per workspace | No hard limit |
Performance Limits
| Metric | Description | Limit |
|---|---|---|
| Connection validation time | Time to validate a new connection | Typically 1-5 minutes |
| Schema refresh time | Time to refresh available tables/columns from a source | Varies by source size |
Usage Limits
| SKU | Description | Limit |
|---|---|---|
| Data capacity | Amount of data that can be imported through sources | Based on contract |
| Import frequency | How often imports sync from sources | Every 24 hours |
For specific limits related to your contract, contact your Customer Success Manager.
FAQ
What's the difference between a Source and a Connection?
What's the difference between a Source and a Connection?
A Source refers to the external platform type (e.g., BigQuery, Snowflake), while a Connection is a specific configured link to that platform with your credentials. You might have one BigQuery source type but multiple BigQuery connections to different projects.
Can I connect to multiple data warehouses?
Can I connect to multiple data warehouses?
Yes, you can create connections to multiple different data warehouse platforms simultaneously. For example, you can have active connections to both BigQuery and Snowflake, and create imports from either.
What permissions does Permutive need on my data warehouse?
What permissions does Permutive need on my data warehouse?
Permutive requires read-only access to the specific tables you want to import. We recommend creating a dedicated service account with minimal permissions scoped to only the necessary datasets or schemas.
Is my data copied to Permutive's servers?
Is my data copied to Permutive's servers?
When you create an import from a source, Permutive reads and processes the specified data to make it available for audience building. The data is processed according to your import configuration and Permutive’s data handling policies. For composable setups, data may remain in your cloud environment.
Can I delete a connection?
Can I delete a connection?
Deleting connections is not currently available in the dashboard but is coming soon. If you need to delete a connection, please contact Permutive support. Note that deleting a connection will cause any imports using that connection to fail.
Can I edit or rotate credentials for an existing connection?
Can I edit or rotate credentials for an existing connection?
Editing connections is not currently supported. If something is wrong with a connection or credentials need to be updated, you’ll need to create a new connection with the correct details and migrate your imports to the new connection.
What happens if my source platform has an outage?
What happens if my source platform has an outage?
If your source platform is unavailable, scheduled imports will fail and retry according to the configured retry policy. Connection status may not change immediately, but you can check import run history to see if syncs are failing.
Can I use connections for exports?
Can I use connections for exports?
Currently, Sources only support importing data into Permutive. Exporting data back to your data warehouse or storage is planned for a future release.
What happens if my source schema changes?
What happens if my source schema changes?
Permutive can detect and apply certain types of schema changes to an existing import:
- Add new columns to an existing import and choose which of the new columns to include
- See detected changes flagged as Supported (can be accepted from the dashboard) or Unsupported (require reverting the change at source)
Changelog
For detailed changelog information, visit our
Changelog.