> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcollate.io/llms.txt
> Use this file to discover all available pages before exploring further.

# ThoughtSpot Hybrid Runner | Collate Integration Guide

> Learn how to configure and use the ThoughtSpot connector in Collate using the Hybrid Runner. Deploy the ingestion agent in your environment for secure, private network metadata extraction.

export const MetadataIngestionUi = ({connector, selectServicePath, addNewServicePath, serviceConnectionPath}) => {
  return <>
   <p>
      To ingest metadata from your sources, you need to create a service connection.
      The service connects your source system with Collate. Once you create
      a service, you can use it to configure your ingestion workflows.<br />
      <br />
      To create a service connection and ingest your metadata, follow the steps below:
  </p>
  <Steps>
    <Step title="Select the Service">
    <ol>
          <li>
            On the left navigation bar, click <strong>Settings</strong>.
          </li>
          <li>
            On the next page, click <strong>Services</strong>, and then select the service.
            <img src="/public/images/connectors/visit-services-page.png" alt="Visit Services Page" />
          </li>
    </ol>
   </Step>


   <Step title="Create a New Service">

       To add a new service connection, click <strong>Add New Service</strong>.
      <img src="/public/images/connectors/create-new-service.png" alt="Create a new Service" />


   </Step>


     <Step title="Select the Connector">
       Select <strong>{connector}</strong> as the service type and click <strong>Next</strong>.


       {selectServicePath && <img src={selectServicePath} alt="Select Service" />}
   </Step>


   <Step title="Name and Describe your Service">
       Enter a unique, descriptive <strong>Service Name</strong> and <strong>Description</strong>.
       <ul>
         <li><strong>Service Name</strong>: Collate identifies services by their service name. Enter a name that distinguishes this deployment from other services, including other {connector} services you are ingesting metadata from.</li>
       </ul>


       <Note>
           The service name cannot be changed after it is set.
       </Note>


       {addNewServicePath && <img src={addNewServicePath} alt="Add New Service" />}
   </Step>


   <Step title="Configure the Service Connection">
       Set up the connection settings required for {connector}. <br /><br />

       Configure the following connection options to set up the service and start ingesting metadata from your sources. The right-hand panel displays help documentation for the selected connection type in the product UI

       {serviceConnectionPath && <img src={serviceConnectionPath} alt="Configure Service connection" />}
   </Step>
   </Steps>
   </>;
};

export const ConnectorDetailsHeader = ({name, icon, stage, availableFeatures, unavailableFeatures = [], availableFeaturesCollate = []}) => {
  const showSubHeading = availableFeatures?.length > 0 || unavailableFeatures?.length > 0 || availableFeaturesCollate?.length > 0;
  const totalAvailableFeatures = [...availableFeatures || [], ...availableFeaturesCollate || []];
  return <div className="container">
      <div className="Heading">
        <div className="flex items-center gap-3">
          {icon && <div className="IconContainer">
              <img src={icon} alt={name} noZoom className="ConnectorIcon" />
            </div>}
          <h1 className="ConnectorName">{name}</h1>
          <span className={`StageBadge ${stage === 'PROD' ? 'prod' : 'beta'}`}>
            {stage}
          </span>
        </div>
      </div>
      {showSubHeading && <div className="SubHeading">
          <div className="FeaturesHeading">Feature List</div>
          <div className="FeaturesList">
            {totalAvailableFeatures.map(feature => <div className="FeatureTag AvailableFeature" key={feature}>
                ✓ {feature}
              </div>)}
            {unavailableFeatures.map(feature => <div className="FeatureTag UnavailableFeature" key={feature}>
                ✕ {feature}
              </div>)}
          </div>
        </div>}
    </div>;
};

<ConnectorDetailsHeader icon="/public/images/connectors/thoughtspot.webp" name="ThoughtSpot" stage="PROD" availableFeatures={["Dashboards", "Charts", "Owners", "Datamodels", "Lineage"]} unavailableFeatures={["Projects"]} />

In this section, we provide guides and references to use the ThoughtSpot connector.
Configure and schedule ThoughtSpot metadata and profiler workflows from the Collate UI:

* [Requirements](#requirements)
* [Connection Details](#connection-details)
* [Metadata Ingestion](#metadata-ingestion)
* [Lineage](#lineage)
* [Troubleshooting](/connectors/dashboard/thoughtspot/troubleshooting)

## Requirements

To access the ThoughtSpot APIs and import liveboards, charts, and data models from ThoughtSpot into OpenMetadata, you need appropriate permissions on your ThoughtSpot instance.

### ThoughtSpot Account Setup and Permissions

1. **Authentication Setup**
   ThoughtSpot supports multiple authentication methods:
   * **Basic Authentication:** Username and password authentication. The user should have appropriate permissions to read metadata from ThoughtSpot.
   * **API Token Authentication:** Use ThoughtSpot API tokens for authentication. Generate API tokens from your ThoughtSpot instance.
2. **API Permissions**
   Ensure your ThoughtSpot user or service account has the following permissions:
   * Read access to liveboards and answers
   * Read access to worksheets and data models
   * Access to metadata APIs
   * Export permissions for TML (ThoughtSpot Modeling Language) data
3. **Multi-tenant Configuration (Optional)**
   If you're using ThoughtSpot Cloud with multiple organizations:
   * Set the `Organization ID` parameter to specify which organization to connect to (only for ThoughtSpot Cloud).

<Tip>
  * For lineage extraction, ensure TML (ThoughtSpot Modeling Language) export is enabled for your user.
</Tip>

## Connection Details

* **Host and Port:**
  The URL of your ThoughtSpot instance.
  Examples:
  * Cloud: `https://my-company.thoughtspot.cloud`
  * On-premise: `https://thoughtspot.company.com`
  * Local: `https://localhost`
    If running ingestion in Docker and ThoughtSpot is on `localhost`, use `host.docker.internal`.
* **Authentication:**
  Choose one of the following:
  * **Basic Authentication:**
    * Username: Your ThoughtSpot username
    * Password: Your ThoughtSpot password
  * **API Token Authentication:**
    * API Token: Your ThoughtSpot API token
* **API Version:**
  The ThoughtSpot API version to use for metadata extraction.
  * `v1`: Legacy API version (callosum endpoints)
  * `v2`: Current API version (recommended, default)
* **Organization ID:**
  For multi-tenant ThoughtSpot Cloud deployments.
  * Leave empty for single-tenant
  * Set to your org ID for multi-tenant

## Metadata Ingestion

<MetadataIngestionUi connector={"ThoughtSpot"} selectServicePath={"/public/images/connectors/thoughtspot/select-service.png"} addNewServicePath={"/public/images/connectors/thoughtspot/add-new-service.png"} serviceConnectionPath={"/public/images/connectors/thoughtspot/service-connection.png"} />

## Connection Configuration

<Steps>
  <Step title="Test the Connection">
    Once the credentials have been added, click on *Test Connection* and *Save* the changes.

    <img src="https://mintcdn.com/collatedocs/L7psA65ao88vmcRI/public/images/connectors/test-connection.png?fit=max&auto=format&n=L7psA65ao88vmcRI&q=85&s=2133f0d65f18df1e57f69d2cc3bdeff4" alt="Test Connection" width="1494" height="310" data-path="public/images/connectors/test-connection.png" />
  </Step>

  <Step title="Configure Metadata Ingestion">
    After adding and testing the dashboard service, configure the metadata ingestion pipeline. To configure, follow the steps below:

    1. Navigate to **Settings** > **Services** > **Dashboards**.
    2. On the **Dashboards** services page, click the service you've added.
    3. Go to the **Agents** tab, and then click **Add Agent** > **Add Metadata Agent**.
    4. Configure the ingestion details. See [Metadata Ingestion Options](#metadata-ingestion-options).
           <img src="https://mintcdn.com/collatedocs/e3_660tPQy7v87Kt/public/images/connectors/configure-metadata-ingestion-dashboards.png?fit=max&auto=format&n=e3_660tPQy7v87Kt&q=85&s=f59296fc8494397a8c303e5596a36dae" alt="Configure Metadata Ingestion" width="1151" height="1537" data-path="public/images/connectors/configure-metadata-ingestion-dashboards.png" />

    #### Metadata Ingestion Options

    * **Name**: This field is the name of the ingestion pipeline. Customize it or use the generated name.
    * **Dashboard Filter Pattern (Optional)**: Use it to control whether to include dashboards as part of metadata ingestion.
      * **Include**: Explicitly include dashboards by adding comma-separated regular expressions to the 'Include' field. OpenMetadata will include all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be excluded.
      * **Exclude**: Explicitly exclude dashboards by adding comma-separated regular expressions to the 'Exclude' field. OpenMetadata will exclude all dashboards with names matching one or more of the supplied regular expressions. All other dashboards will be included.
    * **Chart Filter Pattern (Optional)**: Use it to control whether to include charts as part of metadata ingestion.
      * **Include**: Explicitly include charts by adding comma-separated regular expressions to the 'Include' field. OpenMetadata will include all charts with names matching one or more of the supplied regular expressions. All other charts will be excluded.
      * **Exclude**: Explicitly exclude charts by adding comma-separated regular expressions to the 'Exclude' field. OpenMetadata will exclude all charts with names matching one or more of the supplied regular expressions. All other charts will be included.
    * **Data Model Filter Pattern (Optional)**: Use it to control whether to include data models as part of metadata ingestion.
      * **Include**: Explicitly include data models by adding comma-separated regular expressions to the 'Include' field. OpenMetadata will include all data models with names matching one or more of the supplied regular expressions. All other data models will be excluded.
      * **Exclude**: Explicitly exclude data models by adding comma-separated regular expressions to the 'Exclude' field. OpenMetadata will exclude all data models with names matching one or more of the supplied regular expressions. All other data models will be included.
    * **Project Filter Pattern**: Filter the dashboards, charts, and data sources by projects. Note that all of them support regex as include or exclude. For example, "My project, My proj.\*, .\*Project".
          <Tip>
            We filter the projects by concatenating the entire project hierarchy using dot notation
            (for example, `Project1.NestedProjectA.OtherProject`).
            Make sure the regex filter pattern accounts for this fully-qualified format.
          </Tip>
    * **Enable Debug Log**: Enable this toggle to use debug-level logging.

    <img src="https://mintcdn.com/collatedocs/tp-XigydVK-rcvyo/public/images/connectors/metadata-ingestion-option-1.png?fit=max&auto=format&n=tp-XigydVK-rcvyo&q=85&s=8ebad39aa6d1bfe048301c394ebfe178" alt="Metadata Ingestion Option 1" width="1164" height="1432" data-path="public/images/connectors/metadata-ingestion-option-1.png" />

    * **Lineage Information (Optional)**: Configure this section to enable lineage between your dashboards and the database tables they are built on. OpenMetadata uses the database service name to match and draw the lineage path from table to dashboard.
      * **Db Service Prefixes**: Enter one or more service path prefixes to tell OpenMetadata where to look for the source tables used by your dashboards. Supported formats:
        * `DBServiceName`—matches all tables in the service
        * `DBServiceName.DatabaseName`—matches tables in a specific database
        * `DBServiceName.DatabaseName.SchemaName`—matches tables in a specific schema
        * `DBServiceName.DatabaseName.SchemaName.TableName`—matches a specific table
    * **Query Parser Configuration**: Controls how OpenMetadata parses SQL queries to extract lineage. Use this to select the SQL parser that best fits your data source's dialect.
      * **Query Parser Type**: Choose the SQL parser for lineage extraction:
        * **Auto** (default): Automatically tries SqlGlot first, falls back to SqlFluff, then SqlParse. Recommended for best results.
        * **SqlGlot**: High-performance parser with good dialect support. Falls back to SqlParse on failure.
        * **SqlFluff**: Comprehensive but slower parser with strong dialect support. Falls back to SqlParse on failure.
    * **Include Current Owners**: Enable this toggle to control whether to include owners for the ingested entity if the owner email matches a user stored in the Collate server as part of metadata ingestion. If the ingested entity already exists and has an owner, the owner will not be overwritten.
    * **Mark Deleted Dashboards**: Optional configuration to soft delete dashboards in OpenMetadata if the source dashboards are deleted. After deleting, all associated entities like lineage and other related data for that dashboard will be deleted.
    * **Mark Deleted Data Models**: Optional configuration to soft delete data models in OpenMetadata if the source data models are deleted. After deleting, all associated entities with that data model will be deleted.
    * **Mark Deleted Charts**: Optional configuration to soft delete charts in OpenMetadata if the source charts are deleted. After deleting, all associated entities with that chart will be deleted.
    * **Include Tags**: Enable this toggle to control whether to include tags in metadata ingestion.
    * **Include Data Models**: Enable this toggle to control whether to include data models as part of metadata ingestion.
    * **Include Draft Dashboard**: Enable this toggle to include draft dashboards. By default, this is enabled.
    * **Include Usage**: Enable this toggle to control whether to include usage data as part of metadata ingestion.
    * **Override Metadata**: Enable this toggle to control whether to override the existing metadata in the Collate server with the metadata fetched from the source. If enabled, the metadata fetched from the source will override the existing metadata in OpenMetadata. If disabled, only fields that have no value in OpenMetadata will be updated. This is applicable for fields like description, tags, owner, and displayName.
    * **Override Lineage**: Enable this toggle to control whether to override the existing lineage in OpenMetadata with the lineage fetched from the source. If enabled, existing lineage will be replaced. If disabled, new lineage edges will be added without removing existing ones.

    <img src="https://mintcdn.com/collatedocs/tp-XigydVK-rcvyo/public/images/connectors/metadata-ingestion-option-2.png?fit=max&auto=format&n=tp-XigydVK-rcvyo&q=85&s=38b1928e7e81e5b584207672acf71669" alt="Metadata Ingestion Option 2" width="1158" height="1399" data-path="public/images/connectors/metadata-ingestion-option-2.png" />
  </Step>

  <Step title="Schedule the Ingestion and Deploy">
    Scheduling can be set up at an hourly, daily, weekly, or manual cadence. The
    timezone is in UTC. Select a Start Date to schedule for ingestion. It is
    optional to add an End Date.

    Review your configuration settings. If they match what you intended,
    click Deploy to create the service and schedule metadata ingestion.

    If something doesn't look right, click the Back button to return to the
    appropriate step and change the settings as needed.

    After configuring the workflow, you can click on Deploy to create the
    pipeline.

    <img src="https://mintcdn.com/collatedocs/piJyXg9wW6Ik1lg-/public/images/connectors/schedule.png?fit=max&auto=format&n=piJyXg9wW6Ik1lg-&q=85&s=f1add591824b44456f0e2ff259a21c6f" alt="Schedule the Workflow" width="2733" height="1083" data-path="public/images/connectors/schedule.png" />
  </Step>

  <Step title="View the Ingestion Pipeline">
    Once the workflow has been successfully deployed, you can view the
    Ingestion Pipeline running from the Service Page.

    <img src="https://mintcdn.com/collatedocs/cOe_QuHYxAbkMtTI/public/images/connectors/view-ingestion-pipeline.png?fit=max&auto=format&n=cOe_QuHYxAbkMtTI&q=85&s=8c754af74f99ee70e714f6f707b827e4" alt="View Ingestion Pipeline" width="2733" height="1271" data-path="public/images/connectors/view-ingestion-pipeline.png" />

    <Tip>
      If [AutoPilot](/how-to-guides/admin-guide/applications/autopilot) is enabled, workflows like usage tracking, data lineage, and similar tasks will be handled automatically. Users don’t need to set up or manage them - AutoPilot takes care of everything in the system.
    </Tip>
  </Step>
</Steps>

**Note:**
Lineage creation requires:

* Database service names to be configured in the lineage information
* Access to TML export functionality
  Enable debug logging to troubleshoot issues:

```bash theme={null}
export LOG_LEVEL=DEBUG
```

This will provide detailed information about API calls, data extraction, and lineage creation.

<Tip>
  When using a **Hybrid Ingestion Runner**, any sensitive credential fields—such as passwords, API keys, or private keys—must reference secrets using the following format:

  ```
  password: secret:/my/database/password
  ```

  This applies **only to fields marked as secrets** in the connection form (these typically mask input and show a visibility toggle icon).
  For a complete guide on managing secrets in hybrid setups, see the [Hybrid Ingestion Runner Secret Management Guide](https://docs.getcollate.io/getting-started/day-1/hybrid-saas/hybrid-ingestion-runner#3.-manage-secrets-securely).
</Tip>

## Troubleshooting

<Columns cols={2}>
  <Card title="ThoughtSpot Troubleshooting" href="/connectors/dashboard/thoughtspot/troubleshooting">
    Learn more about how to troubleshoot common ThoughtSpot connector issues and resolve configuration or ingestion errors.
  </Card>
</Columns>