From add41c151b43dd652eb8b74da78bc4e5a3123153 Mon Sep 17 00:00:00 2001 From: Rick Viscomi Date: Tue, 13 Dec 2022 00:48:59 -0500 Subject: [PATCH 1/2] Update README.md --- README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/README.md b/README.md index 5e090a5..d10af3d 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,15 @@ Summary and non-summary outputs can be controlled using the `--pipeline_type` ar This pipeline can read individual HAR files, or a single file containing a list of HAR file paths. +```shell +# Run the `all` pipeline on both desktop and mobile using their pre-generated manifests. +./run_flex_template.sh all --parameters input_file=gs://httparchive/crawls_manifest/*-Nov_1_2022 +# Run the `combined` pipeline on mobile using its manifest. +./run_flex_template.sh combined --parameters input_file=gs://httparchive/crawls_manifest/android-Nov_1_2022 +# Run the `combined` pipeline on desktop using its individual HAR files (much slower, not encouraged). +./run_flex_template.sh combined --parameters input=gs://httparchive/crawls/chrome-Nov_1_2022 +``` + ## Outputs - GCP DataFlow & Monitoring metrics - TODO: runtime metrics and dashboards From f6424939ca8e4dd117eca9c38fe739967aa1c341 Mon Sep 17 00:00:00 2001 From: Rick Viscomi Date: Tue, 13 Dec 2022 01:01:02 -0500 Subject: [PATCH 2/2] Update README.md --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d10af3d..debcd7e 100644 --- a/README.md +++ b/README.md @@ -59,9 +59,11 @@ This pipeline can read individual HAR files, or a single file containing a list ```shell # Run the `all` pipeline on both desktop and mobile using their pre-generated manifests. -./run_flex_template.sh all --parameters input_file=gs://httparchive/crawls_manifest/*-Nov_1_2022 +./run_flex_template.sh all --parameters input_file=gs://httparchive/crawls_manifest/*-Nov_1_2022.txt + # Run the `combined` pipeline on mobile using its manifest. -./run_flex_template.sh combined --parameters input_file=gs://httparchive/crawls_manifest/android-Nov_1_2022 +./run_flex_template.sh combined --parameters input_file=gs://httparchive/crawls_manifest/android-Nov_1_2022.txt + # Run the `combined` pipeline on desktop using its individual HAR files (much slower, not encouraged). ./run_flex_template.sh combined --parameters input=gs://httparchive/crawls/chrome-Nov_1_2022 ```