> ## Documentation Index
> Fetch the complete documentation index at: https://docs.permutive.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Sources (Beta)

> Set up connections to data warehouses and data lakes and import your data into Permutive

export const NoBadge = () => {
  return <span style={{
    display: 'inline-block',
    padding: '0.125rem 0.5rem',
    borderRadius: '0.25rem',
    fontSize: '0.625rem',
    background: '#F7D0E2',
    color: '#1A1A1A',
    fontWeight: '500'
  }}>
      No
    </span>;
};

export const YesBadge = () => {
  return <span style={{
    display: 'inline-block',
    padding: '0.125rem 0.5rem',
    borderRadius: '0.25rem',
    fontSize: '0.625rem',
    background: '#C7E8F9',
    color: '#1A1A1A',
    fontWeight: '500'
  }}>
      Yes
    </span>;
};

<CardGroup cols={3}>
  <Card title="Guides" href="#guides" icon="book-open" />

  <Card title="Issues" href="#troubleshooting" icon="triangle-exclamation" />

  <Card title="FAQ" href="#faq" icon="circle-question" />
</CardGroup>

## Overview

Sources is the area within the Connectivity suite where you manage your external data platform connections and imports. From Sources, you can establish connections to your data warehouses and data lakes, configure imports from your chosen tables, and bring that data into Permutive for audience building and activation.

<Note>
  **Beta Program**

  We're still in the process of fine-tuning and building out this product, so we're grateful for your participation in our Beta phase. We aim to provide a smooth experience, however, you may encounter some minor hiccups or unexpected behavior as we continue to refine and improve the product. Your feedback is invaluable during this time, so please don't hesitate to share your thoughts and experiences with us.
</Note>

## Why Use Sources?

**Centralize your data connections** — Sources provide a single point of management for all your external data platforms. Rather than managing multiple integration methods, you configure your connections once and reuse them across multiple imports.

**Enable rich audience building** — By connecting your data warehouses and storage systems, you unlock the ability to build cohorts based on data that lives outside of Permutive's behavioral collection. This includes CRM data, purchase history, offline conversions, and any other user data you store externally.

**Maintain security and control** — Source connections use secure authentication methods and allow you to control exactly which data Permutive can access. You choose which tables and columns to import, keeping sensitive data protected.

**Support flexible import types** — A single source connection can power multiple types of imports: user profile data for trait-based targeting, activity data for time-bound events, user identity data for Identity Graph enrichment, and group identity data for household cohorts.

## Concepts

### Definitions

* **Source**: An external data platform (such as a data warehouse or cloud storage) that Permutive connects to for importing data. Sources are configured through the Catalog and managed on the Connections page.

* **Catalog**: The interface within the Connectivity Suite where you browse available source platforms and initiate new connections. The Catalog displays all supported sources with their categories and connection status.

* **Connection**: A configured link between Permutive and a source platform. Connections store authentication credentials and settings needed to access your data. A single platform can have multiple connections (e.g., connections to different BigQuery projects).

* **Import**: A configured data pipeline that pulls specific data from a source connection into Permutive. Multiple imports can be created from a single connection.

### Supported Sources

Permutive supports connections to the following source platforms:

| Platform             | Category       | Description                                                        |
| :------------------- | :------------- | :----------------------------------------------------------------- |
| Google BigQuery      | Data Warehouse | Connect to BigQuery datasets to import tables containing user data |
| Snowflake            | Data Warehouse | Connect to Snowflake databases and schemas for data imports        |
| Amazon S3            | Cloud Storage  | Connect to S3 buckets containing data files                        |
| Google Cloud Storage | Cloud Storage  | Connect to GCS buckets containing data files                       |

<Info>
  We're working towards making more source connection types available. If there is a specific data source you'd like to connect, please contact your Customer Success Manager.
</Info>

## Workflows

### Browsing the Catalog

The Catalog page is your starting point for discovering and connecting to new source platforms. Use the Catalog to:

* Browse available source connectors
* Search and filter platforms by name or category

When you find a platform you want to connect to, click the **Connect** button on that platform's card to begin the connection workflow.

<Note>
  You can create multiple connections to a single platform. Even if you already have a connection to a platform (e.g., BigQuery), you can create additional connections to other BigQuery projects or datasets.
</Note>

### Creating a Source Connection

To create a new source connection:

1. Navigate to the **Catalog** page within Connectivity
2. Find and click on the platform you want to connect to
3. Click the **Connect** button
4. Provide a descriptive name for your connection
5. Enter the required authentication credentials for the platform
6. Configure any additional connection settings
7. Click **Save** to create the connection

Once created, your connection will appear on the **Connections** page with a "Processing" status while Permutive validates the credentials. After processing completes, the status changes to "Active" and the connection is ready for use.

### Managing Connections

The **Connections** page provides a centralized view of all your source and destination connections. From here you can:

* View connection status (Active, Inactive, Processing, Deprecated)
* See which platform and type each connection belongs to
* Access connection details and settings

| Column  | Description                                                       |
| :------ | :---------------------------------------------------------------- |
| Name    | The name you provided when creating the connection                |
| Source  | The platform name (e.g., BigQuery, Snowflake)                     |
| Type    | The connection type (currently "Source" for imports)              |
| Created | Date the connection was created                                   |
| Status  | Current availability: Active, Inactive, Processing, or Deprecated |

### Connection Statuses

Source connections have one of four statuses:

* **Active**: The connection is available and can be used to create imports
* **Inactive**: The connection has been disabled and cannot be used for imports
* **Processing**: A new connection is being validated. This typically completes within a few minutes
* **Deprecated**: The connection is no longer supported or has been retired

### Import Types

Once a source connection is active, you can create imports to bring different types of data into Permutive:

* **User Profile Data** — Import static user attributes such as demographics, subscription tiers, or CRM segments for trait-based cohort building and targeting.
* **User Activity Data** — Import time-stamped event or behavioral data such as purchase history or content interactions for time-bound audience building.
* **Identity Graph Data** — Import user identity mappings and household graphs to enrich Permutive's Identity Graph with identifiers and group relationships from your data warehouse. See [Importing User Identity](/guides/signals/identity/importing-user-identity) and [Importing User Group Memberships](/guides/signals/identity/importing-user-group-memberships) for step-by-step guides.

## Guides

Step-by-step instructions for working with Sources.

<CardGroup cols={2}>
  <Card title="Connecting to BigQuery" icon="database" href="/guides/connectivity/sources/connecting-to-bigquery">
    Set up a connection to Google BigQuery
  </Card>

  <Card title="Connecting to Snowflake" icon="snowflake" href="/guides/connectivity/sources/connecting-to-snowflake">
    Set up a connection to Snowflake
  </Card>

  <Card title="Connecting to Amazon S3" icon="aws" href="/guides/connectivity/sources/connecting-to-amazon-s3">
    Set up a connection to Amazon S3
  </Card>

  <Card title="Connecting to Google Cloud Storage" icon="google" href="/guides/connectivity/sources/connecting-to-gcs">
    Set up a connection to Google Cloud Storage
  </Card>

  <Card title="Creating an Import" icon="download" href="/guides/connectivity/sources/creating-an-import">
    Create and configure a data import from an active connection
  </Card>

  <Card title="Actioning Schema Updates (Beta)" icon="arrows-rotate" href="/guides/connectivity/sources/actioning-updates-to-your-source-schema">
    Add new columns to an existing import when your source schema changes
  </Card>
</CardGroup>

## Troubleshooting

The following issues may occur when working with Sources:

<AccordionGroup>
  <Accordion title="Connection stuck in Processing status">
    If your connection remains in "Processing" status for more than 15 minutes:

    * Verify that the authentication credentials are correct
    * Check that the source platform is accessible and not experiencing outages
    * Ensure any required network permissions (IP allowlisting, firewall rules) are configured

    **Solution**: Try creating the connection again with verified credentials. If the issue persists, contact [Support](mailto:support@permutive.com) with your connection details.
  </Accordion>

  <Accordion title="Connection shows as Inactive unexpectedly">
    A connection may become inactive if:

    * Authentication credentials have expired or been revoked
    * The source platform configuration has changed

    **Solution**: If a connection becomes inactive, you'll need to create a new connection with valid credentials. Connections cannot be reactivated once they become inactive.
  </Accordion>

  <Accordion title="Cannot see tables or data in source">
    If you can connect but cannot see expected tables:

    * Verify the service account or user has read permissions on the specific tables
    * Check that you're connecting to the correct project, dataset, or schema
    * Confirm the tables contain data and are not empty

    **Solution**: Review and update permissions in your source platform, ensuring the Permutive service account has appropriate read access.
  </Accordion>

  <Accordion title="Authentication failed error">
    Authentication failures typically occur due to:

    * Incorrect credentials (wrong password, expired token, invalid key file)
    * Missing or incorrect project/account identifiers
    * Service account not enabled or activated

    **Solution**: Double-check all credential fields, regenerate credentials if needed, and ensure the service account is properly configured in your source platform.
  </Accordion>

  <Accordion title="Platform not available in Catalog">
    If you don't see your desired platform in the Catalog:

    * The connector may not yet be supported
    * The connector may be in limited availability or beta
    * Your workspace may not have access to certain connectors

    **Solution**: Contact your Customer Success Manager to inquire about connector availability and request access if needed.
  </Accordion>
</AccordionGroup>

## Environment Compatibility

#### Core Product

Sources functionality is available in the Permutive Dashboard:

| Functionality          | Web Dashboard | API         |
| :--------------------- | :------------ | :---------- |
| Browse Catalog         | <YesBadge />  | <NoBadge /> |
| Create connections     | <YesBadge />  | <NoBadge /> |
| Manage connections     | <YesBadge />  | <NoBadge /> |
| View connection status | <YesBadge />  | <NoBadge /> |

<Note>
  Source management is currently available only through the Permutive Dashboard. API support for programmatic connection management is planned for future releases.
</Note>

#### Supported Source Platforms

| Platform             | Availability                     | Notes                                                    |
| :------------------- | :------------------------------- | :------------------------------------------------------- |
| Google BigQuery      | <YesBadge /> Generally Available | Full support for standard and partitioned tables         |
| Snowflake            | <YesBadge /> Generally Available | Supports all Snowflake cloud providers (AWS, Azure, GCP) |
| Amazon S3            | <YesBadge /> Generally Available | Supports CSV and Parquet files                           |
| Google Cloud Storage | <YesBadge /> Generally Available | Direct GCS bucket access                                 |

## Dependencies

Sources rely on the following products and features being configured for your organization:

| Dependency             | Required | Description                                                                                                         |
| :--------------------- | :------- | :------------------------------------------------------------------------------------------------------------------ |
| Connectivity Suite     | ✓        | Sources are part of the Connectivity Suite. Access to Connectivity must be enabled for your workspace.              |
| Source Platform Access | ✓        | You must have appropriate permissions and credentials in your source platform to establish connections.             |
| Imports                | \~       | While not required for creating connections, Imports are needed to actually bring data from Sources into Permutive. |

## Limits

Sources adhere to the following product specifications and limits.

#### Feature Limits

| Feature                   | Description                                                    | Limit         |
| :------------------------ | :------------------------------------------------------------- | :------------ |
| Connections per platform  | Number of connections you can create to a single platform type | No limit      |
| Connections per workspace | Total number of source connections per workspace               | No hard limit |

#### Performance Limits

| Metric                     | Description                                            | Limit                 |
| :------------------------- | :----------------------------------------------------- | :-------------------- |
| Connection validation time | Time to validate a new connection                      | Typically 1-5 minutes |
| Schema refresh time        | Time to refresh available tables/columns from a source | Varies by source size |

#### Usage Limits

| SKU              | Description                                         | Limit             |
| :--------------- | :-------------------------------------------------- | :---------------- |
| Data capacity    | Amount of data that can be imported through sources | Based on contract |
| Import frequency | How often imports sync from sources                 | Every 24 hours    |

<Info>
  For specific limits related to your contract, contact your Customer Success Manager.
</Info>

## FAQ

<AccordionGroup>
  <Accordion title="What's the difference between a Source and a Connection?">
    A **Source** refers to the external platform type (e.g., BigQuery, Snowflake), while a **Connection** is a specific configured link to that platform with your credentials. You might have one BigQuery source type but multiple BigQuery connections to different projects.
  </Accordion>

  <Accordion title="Can I connect to multiple data warehouses?">
    Yes, you can create connections to multiple different data warehouse platforms simultaneously. For example, you can have active connections to both BigQuery and Snowflake, and create imports from either.
  </Accordion>

  <Accordion title="What permissions does Permutive need on my data warehouse?">
    Permutive requires read-only access to the specific tables you want to import. We recommend creating a dedicated service account with minimal permissions scoped to only the necessary datasets or schemas.
  </Accordion>

  <Accordion title="Is my data copied to Permutive's servers?">
    When you create an import from a source, Permutive reads and processes the specified data to make it available for audience building. The data is processed according to your import configuration and Permutive's data handling policies. For composable setups, data may remain in your cloud environment.
  </Accordion>

  <Accordion title="Can I delete a connection?">
    Deleting connections is not currently available in the dashboard but is coming soon. If you need to delete a connection, please contact [Permutive support](mailto:support@permutive.com). Note that deleting a connection will cause any imports using that connection to fail.
  </Accordion>

  <Accordion title="Can I edit or rotate credentials for an existing connection?">
    Editing connections is not currently supported. If something is wrong with a connection or credentials need to be updated, you'll need to create a new connection with the correct details and migrate your imports to the new connection.
  </Accordion>

  <Accordion title="What happens if my source platform has an outage?">
    If your source platform is unavailable, scheduled imports will fail and retry according to the configured retry policy. Connection status may not change immediately, but you can check import run history to see if syncs are failing.
  </Accordion>

  <Accordion title="Can I use connections for exports?">
    Currently, Sources only support importing data into Permutive. Exporting data back to your data warehouse or storage is planned for a future release.
  </Accordion>

  <Accordion title="What happens if my source schema changes?">
    Permutive can detect and apply certain types of schema changes to an existing import:

    * **Add new columns** to an existing import and choose which of the new columns to include
    * See detected changes flagged as **Supported** (can be accepted from the dashboard) or **Unsupported** (require reverting the change at source)

    Unsupported changes — removing, renaming, reordering, or changing the data type of existing columns — must be reverted at source. Leaving unsupported changes in place can cause cohorts referencing the affected import to stop functioning correctly.

    For the end-to-end workflow, detection details, and CSV vs Parquet differences, see [Actioning Updates to Your Source Schema](/guides/connectivity/sources/actioning-updates-to-your-source-schema).
  </Accordion>
</AccordionGroup>

## Changelog

<Info>
  For detailed changelog information, visit our
  [Changelog](https://changelog.permutive.com/).
</Info>