Skip to content

chore: Update READMEs of crates to be more consistent #17691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 22, 2025
Merged

chore: Update READMEs of crates to be more consistent #17691

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
3663cd9
chore: Update READMEs of crates to be more consistent
Jefffrey Sep 21, 2025
ae03230
Add some more Apache project links
Jefffrey Sep 21, 2025
8bd3d98
Minor formatting
Jefffrey Sep 21, 2025
3faefab
Formatting
Jefffrey Sep 21, 2025
e64000c
Update datafusion/pruning/README.md
Jefffrey Sep 21, 2025
c012f56
suggestion
Jefffrey Sep 21, 2025
940c0ed
formatting
Jefffrey Sep 21, 2025
4b116ea
formatting
Jefffrey Sep 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions datafusion-cli/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,15 @@

<!-- Note this file is included in the crates.io page as well https://crates.io/crates/datafusion-cli -->

# DataFusion Command-line Interface
# Apache DataFusion Command-line Interface

[DataFusion](https://datafusion.apache.org/) is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

DataFusion CLI (`datafusion-cli`) is a small command line utility that runs SQL queries using the DataFusion engine.

[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/

# Frequently Asked Questions

## Where can I find more information?
Expand Down
2 changes: 1 addition & 1 deletion datafusion/catalog-listing/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-catalog-listing"
description = "datafusion-catalog-listing"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
8 changes: 4 additions & 4 deletions datafusion/catalog-listing/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
under the License.
-->

# DataFusion catalog-listing
# Apache DataFusion Catalog Listing
Copy link
Contributor

@alamb alamb Sep 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Apache DataFusion Catalog Listing
# Apache DataFusion Catalog Listing(Table)

Copy link
Contributor Author

@Jefffrey Jefffrey Sep 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the significance of the parentheses here

Copy link
Contributor

@alamb alamb Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry -- I was trying to point out that what this crate really holds is the ListingTable implementation

Copy link
Contributor

@alamb alamb Sep 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor, no changes needed


[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion with [ListingTable], an implementation
of [TableProvider] based on files in a directory (either locally or on remote
Expand All @@ -29,8 +29,8 @@ Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[listingtable]: https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html
[tableprovider]: https://docs.rs/datafusion/latest/datafusion/datasource/trait.TableProvider.html
[`datafusion`]: https://crates.io/crates/datafusion
2 changes: 1 addition & 1 deletion datafusion/catalog/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-catalog"
description = "datafusion-catalog"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
7 changes: 4 additions & 3 deletions datafusion/catalog/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion Catalog
# Apache DataFusion Catalog

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides catalog management functionality, including catalogs, schemas, and tables.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
7 changes: 4 additions & 3 deletions datafusion/common-runtime/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion Common Runtime
# Apache DataFusion Common Runtime

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides common utilities.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
7 changes: 4 additions & 3 deletions datafusion/common/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion Common
# Apache DataFusion Common

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides common data types and utilities.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
13 changes: 5 additions & 8 deletions datafusion/core/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,12 @@
under the License.
-->

# DataFusion Core
<!--
Note the main crates.io landing page https://crates.io/crates/datafusion
uses the workspace README.md file, not this file
-->

DataFusion is an extensible query execution framework, written in Rust,
that uses Apache Arrow as its in-memory format.
# Apache DataFusion Core

This crate contains the main entry points and high level DataFusion APIs such as
`SessionContext`, `DataFrame` and `ListingTable`.

For more information, please see:

- [DataFusion Website](https://datafusion.apache.org)
- [DataFusion API Docs](https://docs.rs/datafusion/latest/datafusion/)
2 changes: 1 addition & 1 deletion datafusion/datasource-avro/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-datasource-avro"
description = "datafusion-datasource-avro"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
10 changes: 6 additions & 4 deletions datafusion/datasource-avro/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,17 @@
under the License.
-->

# DataFusion datasource
# Apache DataFusion Avro DataSource

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that defines a Avro based file source.
This crate is a submodule of DataFusion that defines an [Apache Avro] based file source.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[apache avro]: https://avro.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
2 changes: 1 addition & 1 deletion datafusion/datasource-csv/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-datasource-csv"
description = "datafusion-datasource-csv"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
7 changes: 4 additions & 3 deletions datafusion/datasource-csv/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion datasource
# Apache DataFusion CSV DataSource

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that defines a CSV based file source.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
2 changes: 1 addition & 1 deletion datafusion/datasource-json/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-datasource-json"
description = "datafusion-datasource-json"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
7 changes: 4 additions & 3 deletions datafusion/datasource-json/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion datasource
# Apache DataFusion JSON DataSource

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that defines a JSON based file source.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
2 changes: 1 addition & 1 deletion datafusion/datasource-parquet/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-datasource-parquet"
description = "datafusion-datasource-parquet"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
10 changes: 6 additions & 4 deletions datafusion/datasource-parquet/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,17 @@
under the License.
-->

# DataFusion datasource
# Apache DataFusion Parquet DataSource

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that defines a Parquet based file source.
This crate is a submodule of DataFusion that defines an [Apache Parquet] based file source.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[apache parquet]: https://parquet.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
2 changes: 1 addition & 1 deletion datafusion/datasource/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@
[package]
name = "datafusion-datasource"
description = "datafusion-datasource"
readme = "README.md"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true
version.workspace = true
Expand Down
7 changes: 4 additions & 3 deletions datafusion/datasource/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion datasource
# Apache DataFusion DataSource

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that defines common DataSource related components like FileScanConfig, FileCompression etc.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
1 change: 1 addition & 0 deletions datafusion/doc/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
name = "datafusion-doc"
description = "Documentation module for DataFusion query engine"
keywords = ["datafusion", "query", "sql"]
readme = "README.md"
version = { workspace = true }
edition = { workspace = true }
homepage = { workspace = true }
Expand Down
7 changes: 4 additions & 3 deletions datafusion/doc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
under the License.
-->

# DataFusion Execution
# Apache DataFusion Documentation

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides structures and macros
for documenting user defined functions.
Expand All @@ -28,5 +28,6 @@ Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
7 changes: 4 additions & 3 deletions datafusion/execution/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion Execution
# Apache DataFusion Execution

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides execution runtime such as the memory pools and disk manager.

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
1 change: 1 addition & 0 deletions datafusion/expr-common/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
name = "datafusion-expr-common"
description = "Logical plan and expression representation for DataFusion query engine"
keywords = ["datafusion", "logical", "plan", "expressions"]
readme = "README.md"
version = { workspace = true }
edition = { workspace = true }
homepage = { workspace = true }
Expand Down
7 changes: 4 additions & 3 deletions datafusion/expr-common/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
under the License.
-->

# DataFusion Logical Plan and Expressions
# Apache DataFusion Common Logical Plan and Expressions

[DataFusion][df] is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.
[Apache DataFusion] is an extensible query execution framework, written in Rust, that uses [Apache Arrow] as its in-memory format.

This crate is a submodule of DataFusion that provides common logical expressions

Most projects should use the [`datafusion`] crate directly, which re-exports
this module. If you are already using the [`datafusion`] crate, there is no
reason to use this crate directly in your project as well.

[df]: https://crates.io/crates/datafusion
[apache arrow]: https://arrow.apache.org/
[apache datafusion]: https://datafusion.apache.org/
[`datafusion`]: https://crates.io/crates/datafusion
Loading