From 0630834071d896b1c23e23dd3856ed8bc6bf68f9 Mon Sep 17 00:00:00 2001 From: Aimee Barciauskas Date: Wed, 10 Jan 2024 11:34:10 -0800 Subject: [PATCH 1/2] Update index.md --- docs/composition/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/composition/index.md b/docs/composition/index.md index a766a41b..6a250d12 100644 --- a/docs/composition/index.md +++ b/docs/composition/index.md @@ -4,7 +4,7 @@ A recipe describes the steps to transform archival source data in one format / location into analysis-ready, cloud-optimized (ARCO) data in another format / -location. Technically, a recipe is as a set of composite +location. Technically, a recipe is a composite of [Apache Beam transforms](https://beam.apache.org/documentation/programming-guide/#transforms) applied to the data collection associated with a {doc}`file pattern `. To write a recipe: From c4892df4e6bb6ead9f3762d6a35d68999967d140 Mon Sep 17 00:00:00 2001 From: Aimee Barciauskas Date: Wed, 10 Jan 2024 11:39:38 -0800 Subject: [PATCH 2/2] Update file_patterns.md --- docs/composition/file_patterns.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/composition/file_patterns.md b/docs/composition/file_patterns.md index 55b31aed..c38f5ddf 100644 --- a/docs/composition/file_patterns.md +++ b/docs/composition/file_patterns.md @@ -207,8 +207,8 @@ pattern[index] ## From file pattern to `PCollection` As covered in {doc}`index`, a recipe is composed of a sequence of Apache Beam transforms. -The data collection that Apache Beam transforms operates on is a -[`PCollection`](https://beam.apache.org/documentation/programming-guide/#pcollections). +The data Apache Beam transforms operate on are +[`PCollections`](https://beam.apache.org/documentation/programming-guide/#pcollections). Therefore, we bring the contents of a `FilePattern` into a recipe, we pass the index:url pairs generated by the file pattern's ``items()`` method into Beam's `Create` constructor as follows: