Skip to content

Commit f6f48d4

Browse files
alambadamreeve
andauthored
Improve introduction / overview, add more links to spec and implementation status (#125)
* Improve introduction / overview, add more links to spec and implementation status * Update content/en/docs/Overview/_index.md Co-authored-by: Adam Reeve <adreeve@gmail.com> --------- Co-authored-by: Adam Reeve <adreeve@gmail.com>
1 parent 13be189 commit f6f48d4

3 files changed

Lines changed: 26 additions & 17 deletions

File tree

‎content/en/_index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ It provides high performance compression and encoding schemes to handle complex
1818

1919

2020
{{< blocks/section color="white" type="row">}}
21-
{{% blocks/feature icon="fab fa-jira" title="Parquet Specification" url="https://github.com/apache/parquet-format" %}}
22-
Browse the Parquet Specification
21+
{{% blocks/feature icon="fab fa-jira" title="Documentation" url="docs" %}}
22+
Browse project documentation including the format specification.
2323
{{% /blocks/feature %}}
2424

2525
{{% blocks/feature icon="fab fa-github" title="Contributions welcome!" url="https://github.com/apache/parquet-java" %}}

‎content/en/docs/Overview/_index.md‎

Lines changed: 17 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -9,37 +9,40 @@ description: >
99
Apache Parquet is an open source, column-oriented data file format designed for efficient data storage and retrieval.
1010
It provides high performance compression and encoding schemes to handle complex data in bulk and is supported in many programming language and analytics tools.
1111

12-
This documentation contains information about both the [parquet-java](https://github.com/apache/parquet-java) and [parquet-format](https://github.com/apache/parquet-format) repositories.
1312

14-
### parquet-format
13+
### parquet-format (Specification)
1514

16-
The parquet-format repository hosts the official specification of the Apache Parquet file format, defining how data is structured and stored. This specification, along with Thrift metadata definitions and other crucial components, is essential for developers to effectively read and write Parquet files. The parquet-format project specifically contains the format specifications needed to understand and properly utilize Parquet files.
15+
The [parquet-format] repository hosts the official specification of the Parquet file format, defining how data is structured and stored. This specification, along with the [parquet.thrift] Thrift metadata definitions, is necessary for developing software to effectively read and write Parquet files.
1716

18-
As a repository focused on specification, the parquet-format repository does not contain source code.
17+
Note that the parquet-format repository does not contain source code for libraries to read or write Parquet files, but rather the formal definitions and documentation of the file format itself.
1918

19+
[parquet-format]: https://github.com/apache/parquet-format
20+
[parquet.thrift]: https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift
2021

2122
### parquet-java
2223

23-
The parquet-java (formerly named 'parquet-mr') repository is part of the Apache Parquet project and specifically focuses on providing Java tools for handling the Parquet file format. Essentially, this repository includes all the necessary Java libraries and modules that allow developers to read and write Apache Parquet files.
24+
The [parquet-java] (formerly named `parquet-mr`) repository is part of the Apache Parquet project and contains:
25+
* Java libraries to read and write Parquet files in Java applications.
26+
* Utilities and APIs for working with Parquet files, including tools for data import/export, schema management, and data conversion.
2427

25-
The parquet-java repository contains an implementation of the Apache Parquet format. There are a number of other Parquet format implementations, which are listed below.
26-
27-
Included in parquet-java:
28-
* Java Implementation: It contains the core Java implementation of the Apache Parquet format, making it possible to use Parquet files in Java applications.
29-
30-
* Utilities and APIs: It provides various utilities and APIs for working with Apache Parquet files, including tools for data import/export, schema management, and data conversion.
28+
Note that there are a number of other implementations of the Parquet format, some of which are listed below.
3129

30+
[parquet-java]: https://github.com/apache/parquet-java
3231

3332
### Other Clients / Libraries / Tools
3433

3534
The Parquet ecosystem is rich and varied, encompassing a wide array of tools, libraries, and clients, each offering different levels of feature support. It's important to note that not all implementations support the same features of the Parquet format. When integrating multiple Parquet implementations within your workflow, it is crucial to conduct thorough testing to ensure compatibility and performance across different platforms and tools.
3635

37-
Here is a non-exhaustive list of Parquet implementations:
36+
You can find more information about the feature support of various Parquet implementations on the [implementation status] page.
37+
38+
[implementation status]: /docs/file-format/implementationstatus
39+
40+
Here is a non-exhaustive list of open source Parquet implementations:
3841

3942
* [Parquet-java](https://github.com/apache/parquet-java)
4043
* [Parquet C++, a subproject of Arrow C++](https://github.com/apache/arrow/tree/main/cpp/src/parquet) ([documentation](https://arrow.apache.org/docs/cpp/parquet.html))
41-
* [Parquet Go, a subproject for Arrow Go](https://github.com/apache/arrow-go/tree/main/parquet) ([documentation](https://github.com/apache/arrow-go/tree/main/parquet))
42-
* [Parquet Rust](https://github.com/apache/arrow-rs/blob/main/parquet/README.md)
44+
* [Parquet Go, a subproject of Arrow Go](https://github.com/apache/arrow-go/tree/main/parquet) ([documentation](https://github.com/apache/arrow-go/tree/main/parquet))
45+
* [Parquet Rust, a subproject of Arrow Rust](https://github.com/apache/arrow-rs/blob/main/parquet/README.md)
4346
* [cuDF](https://github.com/rapidsai/cudf)
4447
* [Apache Impala](https://github.com/apache/impala)
4548
* [DuckDB](https://github.com/duckdb/duckdb)

‎content/en/docs/_index.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,10 @@ menu:
88
weight: 2
99
---
1010

11-
Welcome to the documentation for Apache Parquet. Here, you can find information about the Parquet File Format, including specifications and developer resources.
11+
Welcome to the documentation for Apache Parquet.
12+
13+
The specification for the Apache Parquet file format is hosted in the [parquet-format] repository.
14+
The current implementation status of various features can be found in the [implementation status] page.
15+
16+
[parquet-format]: https://github.com/apache/parquet-format
17+
[implementation status]: /docs/file-format/implementationstatus

0 commit comments

Comments
 (0)