Skip to content

Commit

Permalink
docs: fix errors and add 'table' before 'expression'
Browse files Browse the repository at this point in the history
  • Loading branch information
ianmcook authored and cpcloud committed May 8, 2023
1 parent 6e64cd0 commit 096b568
Show file tree
Hide file tree
Showing 8 changed files with 43 additions and 41 deletions.
15 changes: 8 additions & 7 deletions docs/backends/index.md
Expand Up @@ -5,10 +5,10 @@ to inspect or reconfigure the backend used by default.

## String Generating Backends

The first category of backend translate Ibis expressions into string queries.
The first category of backend translate Ibis table expressions into query strings.

The compiler turns each expression into a string query and passes that query to the
database through a driver API for execution.
The compiler turns each table expression into a query string and passes that query
to the database through a driver API for execution.

- [Apache Impala](Impala.md)
- [ClickHouse](ClickHouse.md)
Expand All @@ -17,11 +17,12 @@ database through a driver API for execution.

## Expression Generating Backends

The next category of backends translates ibis expressions into another
system's expressions, for example, SQLAlchemy.
The next category of backends translates Ibis table expressions into another
system's table expression objects, for example, SQLAlchemy.

Instead of generating strings for each expression these backends produce
another kind of expression and typically have high-level APIs for execution.
Instead of generating a query string for each table expression, these backends
produce another kind of table expression object and typically have high-level APIs
for execution.

- [Apache Arrow Datafusion](Datafusion.md)
- [Apache Druid](Druid.md)
Expand Down
15 changes: 8 additions & 7 deletions docs/blog/ibis_substrait_to_duckdb.md
Expand Up @@ -18,9 +18,9 @@ communicating consistently with those backends.
other things) query plans. It's still in its early days, but there is already
nascent support for Substrait in [Apache Arrow](https://arrow.apache.org/docs/dev/cpp/streaming_execution.html#substrait), [DuckDB](https://duckdb.org/docs/extensions/substrait), and [Velox](https://engineering.fb.com/2022/08/31/open-source/velox/).

Ibis supports producing Substrait plans from Ibis expressions, with the help of
the [ibis-substrait](https://github.com/ibis-project/ibis-substrait) library.
Let's take a quick peek at how we might use it for query execution.
Ibis supports producing Substrait plans from Ibis table expressions, with the
help of the [ibis-substrait](https://github.com/ibis-project/ibis-substrait)
library. Let's take a quick peek at how we might use it for query execution.

## Getting started

Expand Down Expand Up @@ -50,7 +50,7 @@ con.execute(
## Query Creation

For our example, we'll build up a query using Ibis but without connecting to our
execution engine (DuckDB). Once we have an Ibis expression, we'll create a
execution engine (DuckDB). Once we have an Ibis table expression, we'll create a
Substrait plan, then execute that plan directly on DuckDB to get results.

To do this, all we need is some knowledge of the schema of the tables we want to
Expand Down Expand Up @@ -164,12 +164,13 @@ topfilms = (
)
```

Now that we have an Ibis expression, it's time for Substrait to enter the scene.
Now that we have an Ibis table expression, it's time for Substrait to enter
the scene.

## Substrait Serialization

We're going to import `ibis_substrait` and compile the `topfilms` expression
into a Substrait plan.
We're going to import `ibis_substrait` and compile the `topfilms` table
expression into a Substrait plan.

```python
from ibis_substrait.compiler.core import SubstraitCompiler
Expand Down
9 changes: 5 additions & 4 deletions docs/getting_started.md
Expand Up @@ -79,9 +79,9 @@ to get the first few rows of the table as a pandas DataFrame.
4 Adelie Torgersen 36.7 19.3 193.0 3450.0 female 2007
```

`execute` takes the existing lazy expression and evaluates it. If we leave it
off, you'll see the Ibis representation of the expression that `execute` will
evaluate (when you're ready!).
`execute` takes the existing lazy table expression and evaluates it. If we
leave it off, you'll see the Ibis representation of the table expression that
`execute` will evaluate (when you're ready!).

```python
>>> penguins.head()
Expand Down Expand Up @@ -110,7 +110,8 @@ Limit[r0, n=5]
For the rest of this intro, we'll turn on interactive mode, which partially
executes queries to give users a preview of the results. There is a small
difference in the way the output is formatted, but otherwise this is the same
as calling `execute()` on the expression with a limit of 10 results returned.
as calling `execute()` on the table expression with a limit of 10 result rows
returned.

```python
>>> ibis.options.interactive = True
Expand Down
2 changes: 1 addition & 1 deletion docs/how_to/sessionize.md
Expand Up @@ -74,6 +74,6 @@ sessionized = (
)
```

Calling `ibis.show_sql(sessionized)` displays the SQL query and can be used to confirm that this Ibis expression does not rely on any join operations.
Calling `ibis.show_sql(sessionized)` displays the SQL query and can be used to confirm that this Ibis table expression does not rely on any join operations.

Calling `sessionized.execute()` should complete in less than a minute, depending on the speed of the internet connection to download the data and the number of CPU cores available to parallelize the processing of this nested query.
2 changes: 1 addition & 1 deletion docs/ibis-for-dplyr-users.ipynb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 5 additions & 6 deletions docs/ibis-for-pandas-users.ipynb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 14 additions & 14 deletions docs/user_guide/design.md
Expand Up @@ -18,12 +18,12 @@
1. The SQL string that generated by the compiler is sent to the database and
executed (this step is skipped for the pandas backend)
1. The database returns some data that is then turned into a pandas DataFrame
by ibis
by Ibis

## Expressions

The main user-facing component of ibis is expressions. The base class of all
expressions in ibis is the [ibis.expr.types.Expr][] class.
The main user-facing component of Ibis is expressions. The base class of all
expressions in Ibis is the [ibis.expr.types.Expr][] class.

Expressions provide the user facing API, most of which is defined in
`ibis/expr/api.py`.
Expand All @@ -32,7 +32,7 @@ Expressions provide the user facing API, most of which is defined in

Ibis's type system consists of a set of rules for specifying the types of
inputs to `ibis.expr.types.Node` subclasses. Upon construction of a `Node`
subclass, ibis performs validation of every input to the node based on the rule
subclass, Ibis performs validation of every input to the node based on the rule
that was used to declare the input.

Rules are defined in `ibis.expr.rules`
Expand All @@ -55,7 +55,7 @@ Examples of expression types include
### The `ibis.expr.types.Node` Class
<!-- prettier-ignore-end -->

`Node` subclasses make up the core set of operations of ibis. Each node
`Node` subclasses make up the core set of operations of Ibis. Each node
corresponds to a particular operation.

Most nodes are defined in the `ibis.expr.operations` module.
Expand Down Expand Up @@ -93,7 +93,7 @@ or column named `base` that defaults to nothing if not provided. The `base`
argument is `None` by default so that the expression will behave as the
underlying database does.

Similar objects are instantiated when you use ibis APIs:
Similar objects are instantiated when you use Ibis APIs:

```python
import ibis
Expand All @@ -118,16 +118,16 @@ type. An example of this is the `greatest` function, which takes the maximum
of all of its arguments. Another example is `CASE` statements, whose `THEN`
expressions determine the output type of the expression.

This allows ibis to provide **only** the APIs that make sense for a particular
This allows Ibis to provide **only** the APIs that make sense for a particular
type, even when an operation yields a different output type depending on its
input. Concretely, this means that you cannot perform operations that don't
make sense, like computing the average of a string column.

## Compilation

The next major component of ibis is the compilers.
The next major component of Ibis is the compilers.

The first few versions of ibis directly generated strings, but the compiler
The first few versions of Ibis directly generated strings, but the compiler
infrastructure was generalized to support compilation of
[SQLAlchemy](https://docs.sqlalchemy.org/en/latest/core/tutorial.html) based
expressions.
Expand All @@ -153,23 +153,23 @@ that will perform this translation.

!!! note "Ibis can target other systems besides SQL"

While ibis was designed with an explicit goal of first-class SQL support,
ibis can target other systems such as pandas.
While Ibis was designed with an explicit goal of first-class SQL support,
Ibis can target other systems such as pandas.

## Execution

Presumably we want to _do_ something with our compiled expressions. This is
where execution comes in.

This is least complex part of ibis, mostly only requiring ibis to correctly
This is least complex part of Ibis, mostly only requiring Ibis to correctly
handle whatever the database hands back.

By and large, the execution of compiled SQL is handled by the database to which
SQL is sent from ibis.
SQL is sent from Ibis.

However, once the data arrives from the database we need to convert that
data to a pandas DataFrame.

The Query class, with its `ibis.sql.client.Query._fetch` method, provides a way
for ibis `ibis.sql.client.SQLClient` objects to do any additional processing
for Ibis `ibis.sql.client.SQLClient` objects to do any additional processing
necessary after the database returns results to the client.
2 changes: 1 addition & 1 deletion docs/user_guide/extending/reduction.ipynb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 096b568

Please sign in to comment.