kumoai.connector.DatabricksConnector

kumoai.connector.DatabricksConnector#

class kumoai.connector.DatabricksConnector[source]#

Bases: Connector

Establishes a connection to a Databricks database.

Authentication requires passing either a client ID and client secret, or a personal access token, to the connector, either via environment variables (DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET, or DATABRICKS_TOKEN), or via keys in the credentials dictionary (client_id and client_secret, or token).

import kumoai

# Either pass `credentials=dict(client_id=..., client_secret=...,
# token=...) or set the 'DATABRICKS_CLIENT_ID' and
# 'DATABRICKS_CLIENT_SECRET' (or 'DATABRICKS_TOKEN') environment
# variables:
connector = kumoai.connector.DatabricksConnector(
    name="<connector_name>",
    host="<databricks_host_name>",
    cluster_id="<databricks_cluster_id>",
    warehouse_id="<databricks_warehouse_id>",
    catalog="<databricks_catalog_name>",
    credentials=credentials,
)

# List all tables:
print(connector.table_names())

# Check whether a table is present:
assert "articles" in connector

# Fetch a source table (both approaches are equivalent):
source_table = connector["articles"]
source_table = connector.table("articles")
Parameters:

  • name (str) – The name of the connector.

  • host (str) – The host name.

  • cluster_id (str) – The cluster ID of this warehouse.

  • warehouse_id (str) – The warehouse ID of this warehous.

  • catalog (str) – The name of the Databricks catalog.

  • credentials (Optional[Dict[str, str]]) – The client ID, client secret, and personal access token that correspond to this Databricks account.

__init__(name, host, cluster_id, warehouse_id, catalog, credentials=None, _bypass_creation=False)[source]#
classmethod get_by_name(name)[source]#

Returns an instance of a named Databricks Connector, including those created in the Kumo UI.

Parameters:

name (str) – The name of the existing connector.

Return type:

Self

Example

>>> import kumoai
>>> connector = kumoai.DatabricksConnector.get_by_name("name")
property name: str#

Returns the name of this connector.

property source_type: DataSourceType#

Returns the data source type accessible by this connector.

has_table(name)#

Returns True if the table exists in this connector, False otherwise.

Parameters:

name (str) – The table name.

Return type:

bool

table(name)#

Returns a SourceTable object corresponding to a source table behind this connector. A source table is a view into the raw data of table name. To use a source table in Kumo, you will need to construct a Table from the source table.

Parameters:

name (str) – The table name.

Raises:

ValueError – if name does not exist in the backing connector.

Return type:

SourceTable

table_names()#

Returns a list of table names accessible through this connector.

Return type:

List[str]

Contents