Load query results
Import SQL query results into Polars for filtering, aggregation, or further DataFrame processing.
Polars · Function
Executes a SQL query through a database connection and returns the results as a Polars DataFrame. Enable batch iteration to process large result sets incrementally.
The function executes a SQL query through an existing connection or cursor and converts the results into a Polars DataFrame. With iter_batches enabled it returns an iterator of DataFrames, while schema_overrides can replace inferred column types and execute_options can be forwarded to the underlying execution call.
polars.read_database(query: str | TextClause | Selectable, connection: ConnectionOrCursor | str, *, iter_batches: bool = False, batch_size: int | None = None, schema_overrides: SchemaDict | None = None, infer_schema_length: int | None = 100, execute_options: dict[str, Any] | None = None) → DataFrame | Iterator[DataFrame]| Name | Type | Description |
|---|---|---|
query | str | TextClause | Selectable | SQL query to execute; with SQLAlchemy, this may be a suitable Selectable, otherwise it is expected to be a string. |
connection | ConnectionOrCursor | str | Connection, cursor, or client object against which the query can execute; a connection string is also accepted by the signature. |
iter_batches | bool = False | Return an iterator of DataFrames, one for each query-result batch. |
batch_size | int | None = None | Batch size used for batch processing. |
schema_overrides | SchemaDict | None = None | Column-to-dtype mapping overriding inferred schema. |
infer_schema_length | int | None = 100 | Number of rows used for schema inference. |
execute_options | dict[str, Any] | None = None | Options forwarded to query execution. |
DataFrame | Iterator[DataFrame]
import polars as pl
import sqlite3
connection = sqlite3.connect(":memory:")
connection.execute("CREATE TABLE people (name TEXT, age INTEGER)")
connection.executemany("INSERT INTO people VALUES (?, ?)", [("Ada", 36), ("Grace", 28)])
frames = pl.read_database(
query="SELECT name, age FROM people",
connection=connection,
iter_batches=True,
batch_size=1,
)
for frame in frames:
print(frame)Creates an in-memory SQLite table, reads its query results in one-row DataFrame batches, and prints each batch.
Import SQL query results into Polars for filtering, aggregation, or further DataFrame processing.
Handle a large query result without requiring all returned rows in one processing step.
Correct or constrain backend-inferred column types during ingestion.
It runs a SQL query using the supplied connection or cursor and returns a DataFrame containing the query results. The query may be a string, or a SQLAlchemy Selectable when using a SQLAlchemy connection.
Follow-up: How would you process a result set incrementally? Set iter_batches=True and optionally choose batch_size, then consume the returned DataFrame iterator.
Set iter_batches=True to receive an iterator whose elements are DataFrames rather than one complete DataFrame. This is intended to make processing large result sets more memory-efficient.
Follow-up: Can batch_size be supplied when iter_batches is false? Yes; the function may construct the final DataFrame internally using batched returns.
schema_overrides is a column-to-dtype mapping that replaces the schema inferred from the query cursor or incoming Arrow data, depending on the backend. execute_options is a dictionary passed through to the underlying query execution method as keyword arguments.
Follow-up: Why might schema_overrides matter? It lets the caller enforce expected Polars dtypes when backend inference is unsuitable.
polars.read_database_uripolars.DataFrame.write_database