kumoai.connector.upload_table

Contents

kumoai.connector.upload_table#

kumoai.connector.upload_table(name, path, auto_partition=True, partition_size_mb=250)[source]#

Synchronously uploads a table located on your local machine to the Kumo data plane. Tables uploaded in this way can be accessed with a FileUploadConnector.

For files larger than 1GB, the table will be automatically partitioned into smaller chunks and uploaded with common prefix that allows FileUploadConnector to union them when reading.

Warning

Uploaded tables must be single files, either in parquet or CSV format. Partitioned tables are not currently supported.

import kumoai
from kumoai.connector import upload_table

# Upload a small table
upload_table(name="users", path="/data/users.parquet")

# Upload a large parquet table (will be automatically partitioned)
upload_table(name="transactions",
             path="/data/large_transactions.parquet")

# Upload a large CSV table (will be automatically partitioned)
upload_table(name="sales", path="/data/large_sales.csv")

# Disable auto-partitioning (will raise error for large files)
upload_table(name="users", path="/data/users.parquet",
             auto_partition=False)
Parameters:

  • name (str) – The name of the table to be uploaded. The uploaded table can be accessed from the FileUploadConnector with this name.

  • path (str) – The full path of the table to be uploaded, on the local machine.

  • auto_partition (bool) – Whether to automatically partition large files (>1GB). If False and file is >1GB, raises ValueError. Supports both Parquet and CSV files.

  • partition_size_mb (int) – The size of each partition in MB. Only used if auto_partition is True.

Return type:

None

Contents