Skip to main content

​
Overview

The Databricks Agent Node provides an AI agent for interacting with Databricks SQL Warehouses and Vector Search indexes. This node enables:
  • Execute SQL statements against Databricks SQL Warehouses
  • Perform similarity searches using Databricks Vector Search
  • Query and analyze data stored in Databricks
  • Access Delta Lake tables and unified analytics platform
  • Support for both inline results and external links for large datasets

​
Configuration Parameters

To set up the Databricks Agent Node, authenticate with your Databricks workspace and configure the agent’s capabilities.

​
Node Configuration

  • Workspace URL: Your Databricks workspace base URL (e.g., https://dbc-xxxx.cloud.databricks.com).
  • Prompt: Describe what you want (your intent) in natural language. The agent figures out how to translate that into SQL or vector search—you don’t need to write the query yourself. Intent-based (recommended):
    Show me the top 10 customers by revenue
    What were total sales in the West region last quarter?
    Find documents in the knowledge base that are similar to “machine learning model deployment”
    You can also give a raw SQL statement or specific instructions if you already know the exact query, but for most use cases describing the goal is enough.
Prompt by intent. Describe the analysis or data you want (e.g. “top 10 customers by revenue”, “documents similar to X”). The agent will choose and run the right SQL or vector search—no need to write queries yourself unless you have a fixed query.
  • Tools Select which tools the Databricks Agent can use:
    Tool nameDescription
    Execute SQLRun SQL statements against SQL Warehouses
    Search Vector IndexPerform similarity searches against vector indexes
    For large result sets, the agent can use EXTERNAL_LINKS disposition to avoid memory issues.
  • System Prompt: Customize the agent’s behavior and query guidelines.
  • Model: Select the language model for the agent (must support function calling).
  • Max Steps: Set the maximum number of steps the agent can take (1-20).
  • Enable Model Fallbacks: Allow automatic fallback to alternative models if the primary model fails.

​
Expected Inputs and Outputs

  • Inputs:
    • The node accepts text input that can be referenced in the prompt using format strings.
  • Outputs:
    • output: The final result from the agent’s SQL queries or vector searches
    • conversation: Detailed log of the agent’s operations

​
Use Case Examples

  1. Data Analytics: Use the agent to query Databricks tables, perform aggregations, and generate analytical insights from your data warehouse.
  2. Vector Search: Configure the agent to perform semantic searches across your vector indexes, finding similar documents or content based on meaning.
  3. Automated Reporting: Let the agent extract data from Databricks and prepare it for reporting, handling complex joins and transformations.

​
Error Handling and Troubleshooting

  • Authentication Issues: If the node fails to authenticate, ensure you’ve connected your Databricks account with OAuth and provided the correct workspace URL.
  • Warehouse Configuration: Make sure the warehouse ID you’re querying exists and is accessible to your account.
  • Model Compatibility: The selected model must support function calling. Switch to a compatible model if you encounter errors.
If you encounter any issues not covered in this documentation, please reach out to our support team for assistance.

​
Relevant Nodes

Agent Node

General-purpose agent for complex tasks

PostgreSQL Query Node

Execute SQL queries against PostgreSQL databases

MySQL Query Node

Execute SQL queries against MySQL databases