Skip to content

Commit

Permalink
Merge pull request #45 from IBM/make-run-samples
Browse files Browse the repository at this point in the history
readme changes for recent makefile sample runs
  • Loading branch information
daw3rd authored May 1, 2024
2 parents a2fe210 + 7babf9b commit 8545c49
Show file tree
Hide file tree
Showing 9 changed files with 28 additions and 16 deletions.
2 changes: 1 addition & 1 deletion transforms/code/code_quality/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ To run the samples, use the following `make` targets
* `run-cli-ray-sample` - runs src/code_quality_transform.py using command line args
* `run-local-ray-sample` - runs src/code_quality_local_ray.py
* `run-s3-ray-sample` - runs src/code_quality_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.
Expand Down
2 changes: 1 addition & 1 deletion transforms/code/malware/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ To run the samples, use the following `make` targets
* `run-local-sample` - runs src/malware_local.py
* `run-local-ray-sample` - runs src/malware_local_ray.py
These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.
For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/code/proglang_select/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ To run the samples, use the following `make` targets
* `run-local-sample` - runs src/proglang_select_local.py
* `run-local-ray-sample` - runs src/proglang_select_local_ray.py
* `run-s3-ray-sample` - runs src/proglang_select_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.
These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.
For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/universal/doc_id/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,9 @@ To run the samples, use the following `make` targets
* `run-local-sample` - runs src/doc_id_local.py
* `run-local-ray-sample` - runs src/doc_id_local_ray.py
* `run-s3-ray-sample` - runs src/doc_id_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/universal/ededup/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ To run the samples, use the following `make` targets
* `run-cli-ray-sample` - runs src/ededup_transform.py using command line args
* `run-local-ray-sample` - runs src/ededup_local_ray.py
* `run-s3-ray-sample` - runs src/ededup_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/universal/fdedup/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,9 @@ To run the samples, use the following `make` targets
* `run-cli-ray-sample` - runs src/fdedup_transform.py using command line args
* `run-local-ray-sample` - runs src/fdedup_local_ray.py
* `run-s3-ray-sample` - runs src/fdedup_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/universal/filter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,9 +259,9 @@ To run the samples, use the following `make` targets
* `run-local-sample` - runs src/filter_local.py
* `run-local-ray-sample` - runs src/filter_local_ray.py
* `run-s3-ray-sample` - runs src/filter_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down
16 changes: 14 additions & 2 deletions transforms/universal/noop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,31 @@ The set of dictionary keys holding [NOOPTransform](src/noop_transform.py)
configuration for values are as follows:

* _noop_sleep_sec_ - specifies the number of seconds to sleep during table transformation.
* _noop_pwd_ - specifies a dummy password not included in metadata.

## Running

### Launched Command Line Options
When running the transform with the Ray launcher (i.e. TransformLauncher),
the following command line arguments are available in addition to
[the options provided by the launcher](../../../data-processing-lib/doc/launcher-options.md).
```
--noop_sleep_sec NOOP_SLEEP_SEC
Sleep actor for a number of seconds while processing the data frame, before writing the file to COS
--noop_pwd NOOP_PWD A dummy password which should be filtered out of the metadata
```
These correspond to the configuration keys described above.

### Running the samples
To run the samples, use the following `make` targets

* `run-cli-ray-sample` - runs src/noop_transform.py using command line args
* `run-local-sample` - runs src/noop_local.py
* `run-local-ray-sample` - runs src/noop_local_ray.py
* `run-s3-ray-sample` - runs src/noop_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down
4 changes: 2 additions & 2 deletions transforms/universal/tokenization/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,9 @@ To run the samples, use the following `make` targets
* `run-cli-ray-sample` - runs src/tokenization_transform.py using command line args
* `run-local-ray-sample` - runs src/tokenization_local_ray.py
* `run-s3-ray-sample` - runs src/tokenization_s3_ray.py
* Requires prior invocation of `make minio-start minio-load` to load data into local minio for S3 access.
* Requires prior invocation of `make minio-start` to load data into local minio for S3 access.

These targes will activate the virtual environment and set up any configuration needed.
These targets will activate the virtual environment and set up any configuration needed.
Use the `-n` option of `make` to see the detail of what is done to run the sample.

For example,
Expand Down

0 comments on commit 8545c49

Please sign in to comment.