Skip to content

Commit

Permalink
Rename crew_aws_batch_monitor() to crew_monitor_aws_batch()
Browse files Browse the repository at this point in the history
  • Loading branch information
wlandau-lilly committed Jan 2, 2024
1 parent 4d87dc7 commit f7f6481
Show file tree
Hide file tree
Showing 11 changed files with 178 additions and 177 deletions.
4 changes: 2 additions & 2 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
S3method(crew_aws_batch_client,crew_aws_batch_debug)
S3method(crew_aws_batch_client,default)
export(crew_aws_batch_launch)
export(crew_aws_batch_monitor)
export(crew_aws_batch_terminate)
export(crew_class_aws_batch_monitor)
export(crew_class_launcher_aws_batch)
export(crew_class_monitor_aws_batch)
export(crew_controller_aws_batch)
export(crew_launcher_aws_batch)
export(crew_monitor_aws_batch)
importFrom(R6,R6Class)
importFrom(crew,crew_assert)
importFrom(crew,crew_class_launcher)
Expand Down
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# crew.cluster 0.0.1.9000

* Use `paws.common::paginate()` to get the full log of a job (#5). Requires `paws.common` >= 0.7.0 due to https://github.com/paws-r/paws/issues/721.
* Rename `crew_aws_batch_monitor()` to `crew_monitor_aws_batch()` for syntactic consistency.

# crew.cluster 0.0.1

Expand Down
70 changes: 35 additions & 35 deletions R/crew_aws_batch_monitor.R → R/crew_monitor_aws_batch.R
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
#' job queue.
#' @param job_definition Character of length 1, name of the AWS Batch
#' job definition. The job definition might or might not exist
#' at the time `crew_aws_batch_monitor()` is called. Either way is fine.
#' at the time `crew_monitor_aws_batch()` is called. Either way is fine.
#' @param log_group Character of length 1,
#' AWS Batch CloudWatch log group to get job logs.
#' The default log group is often "/aws/batch/job", but not always.
Expand All @@ -28,7 +28,7 @@
#' default to `paws.common::get_config()$region`, then to
#' `Sys.getenv("AWS_REGION")` if unsuccessful, then
#' `Sys.getenv("AWS_REGION")`, then `Sys.getenv("AWS_DEFAULT_REGION")`.
crew_aws_batch_monitor <- function(
crew_monitor_aws_batch <- function(
job_queue,
job_definition = paste0(
"crew-aws-batch-job-definition-",
Expand All @@ -43,7 +43,7 @@ crew_aws_batch_monitor <- function(
region <- region %|||% paws.common::get_config()$region
region <- region %|||chr% Sys.getenv("AWS_REGION", unset = "")
region <- region %|||chr% Sys.getenv("AWS_DEFAULT_REGION", unset = "")
out <- crew_class_aws_batch_monitor$new(
out <- crew_class_monitor_aws_batch$new(
job_queue = job_queue,
job_definition = job_definition,
log_group = log_group,
Expand All @@ -60,9 +60,9 @@ crew_aws_batch_monitor <- function(
#' @export
#' @family monitor
#' @description AWS Batch job definition `R6` class
#' @details See [crew_aws_batch_monitor()].
crew_class_aws_batch_monitor <- R6::R6Class(
classname = "crew_class_aws_batch_monitor",
#' @details See [crew_monitor_aws_batch()].
crew_class_monitor_aws_batch <- R6::R6Class(
classname = "crew_class_monitor_aws_batch",
cloneable = FALSE,
private = list(
.job_queue = NULL,
Expand Down Expand Up @@ -320,45 +320,45 @@ crew_class_aws_batch_monitor <- R6::R6Class(
}
),
active = list(
#' @field job_queue See [crew_aws_batch_monitor()].
#' @field job_queue See [crew_monitor_aws_batch()].
job_queue = function() {
.subset2(private, ".job_queue")
},
#' @field job_definition See [crew_aws_batch_monitor()].
#' @field job_definition See [crew_monitor_aws_batch()].
job_definition = function() {
.subset2(private, ".job_definition")
},
#' @field log_group See [crew_aws_batch_monitor()].
#' @field log_group See [crew_monitor_aws_batch()].
log_group = function() {
.subset2(private, ".log_group")
},
#' @field config See [crew_aws_batch_monitor()].
#' @field config See [crew_monitor_aws_batch()].
config = function() {
.subset2(private, ".config")
},
#' @field credentials See [crew_aws_batch_monitor()].
#' @field credentials See [crew_monitor_aws_batch()].
credentials = function() {
.subset2(private, ".credentials")
},
#' @field endpoint See [crew_aws_batch_monitor()].
#' @field endpoint See [crew_monitor_aws_batch()].
endpoint = function() {
.subset2(private, ".endpoint")
},
#' @field region See [crew_aws_batch_monitor()].
#' @field region See [crew_monitor_aws_batch()].
region = function() {
.subset2(private, ".region")
}
),
public = list(
#' @description AWS Batch job definition constructor.
#' @return AWS Batch job definition object.
#' @param job_queue See [crew_aws_batch_monitor()].
#' @param job_definition See [crew_aws_batch_monitor()].
#' @param log_group See [crew_aws_batch_monitor()].
#' @param config See [crew_aws_batch_monitor()].
#' @param credentials See [crew_aws_batch_monitor()].
#' @param endpoint See [crew_aws_batch_monitor()].
#' @param region See [crew_aws_batch_monitor()].
#' @param job_queue See [crew_monitor_aws_batch()].
#' @param job_definition See [crew_monitor_aws_batch()].
#' @param log_group See [crew_monitor_aws_batch()].
#' @param config See [crew_monitor_aws_batch()].
#' @param credentials See [crew_monitor_aws_batch()].
#' @param endpoint See [crew_monitor_aws_batch()].
#' @param region See [crew_monitor_aws_batch()].
initialize = function(
job_queue = NULL,
job_definition = NULL,
Expand Down Expand Up @@ -415,14 +415,14 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description Register a job definition.
#' @details The `register()` method registers a simple
#' job definition using the job definition name and log group originally
#' supplied to [crew_aws_batch_monitor()].
#' supplied to [crew_monitor_aws_batch()].
#' Job definitions created with `$register()` are container-based
#' and use the AWS log driver.
#' For more complicated
#' kinds of jobs, we recommend skipping `register()`: first call
#' <https://www.paws-r-sdk.com/docs/batch_register_job_definition/>
#' to register the job definition, then supply the job definition
#' name to the `job_definition` argument of [crew_aws_batch_monitor()].
#' name to the `job_definition` argument of [crew_monitor_aws_batch()].
#' @return A one-row `tibble` with the job definition name, ARN, and
#' revision number of the registered job definition.
#' @param image Character of length 1, Docker image used for each job.
Expand Down Expand Up @@ -502,7 +502,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description Attempt to deregister the job definition.
#' @details Attempt to deregister the job definition whose name was
#' originally supplied to the `job_definition` argument of
#' [crew_aws_batch_monitor()].
#' [crew_monitor_aws_batch()].
#' @return `NULL` (invisibly).
deregister = function() {
# Covered in tests/interactive/definitions.R
Expand Down Expand Up @@ -549,7 +549,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description Submit a single AWS Batch job to the given job queue
#' under the given job definition.
#' @details This method uses the job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' Any jobs submitted this way are different from the
#' `crew` workers that the `crew` controller starts automatically
#' using the AWS Batch launcher plugin.
Expand Down Expand Up @@ -713,7 +713,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @details This method assumes the job has log driver `"awslogs"`
#' (specifying AWS CloudWatch) and that the log group is the one
#' prespecified in the `log_group` argument of
#' [crew_aws_batch_monitor()]. This method cannot use
#' [crew_monitor_aws_batch()]. This method cannot use
#' other log drivers such as Splunk, and it will fail if the log
#' group is wrong or missing.
#' @return A `tibble` with log information.
Expand Down Expand Up @@ -777,7 +777,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' with the given job definition.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
#' @param status Character vector of job states. Results are limited
Expand Down Expand Up @@ -873,7 +873,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' runnable, starting, or running (not succeeded or failed).
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
active = function() {
Expand All @@ -894,7 +894,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' runnable, starting, or running).
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
inactive = function() {
Expand All @@ -906,7 +906,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"submitted"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
submitted = function() {
Expand All @@ -918,7 +918,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"pending"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
pending = function() {
Expand All @@ -930,7 +930,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"runnable"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
runnable = function() {
Expand All @@ -942,7 +942,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"starting"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
starting = function() {
Expand All @@ -954,7 +954,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"running"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
running = function() {
Expand All @@ -966,7 +966,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"succeeded"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
succeeded = function() {
Expand All @@ -978,7 +978,7 @@ crew_class_aws_batch_monitor <- R6::R6Class(
#' @description List jobs whose status is `"failed"`.
#' @details The output only includes jobs under the
#' job queue and job definition
#' that were supplied through [crew_aws_batch_monitor()].
#' that were supplied through [crew_monitor_aws_batch()].
#' @return A `tibble` with one row per job and columns
#' with job information.
failed = function() {
Expand Down
8 changes: 4 additions & 4 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -97,16 +97,16 @@ str(groups$SecurityGroups[[1L]])

With `crew.aws.batch`, your `crew` controller automatically submits jobs to AWS Batch. These jobs may fail or linger for any number of reasons, which could impede work and increase costs. So before you use `crew_controller_aws_batch()`, please learn how to monitor and terminate AWS Batch jobs manually.

`crew.aws.batch` defines a "monitor" class to help you take control of jobs and job definitions. Create a monitor object with `crew_aws_batch_monitor()`. You will need to supply a job definition name and a job queue name.
`crew.aws.batch` defines a "monitor" class to help you take control of jobs and job definitions. Create a monitor object with `crew_monitor_aws_batch()`. You will need to supply a job definition name and a job queue name.

```{r}
monitor <- crew_aws_batch_monitor(
monitor <- crew_monitor_aws_batch(
job_definition = "YOUR_JOB_DEFINITION_NAME",
job_queue = "YOUR_JOB_QUEUE_NAME"
)
```

The job definition may or may not exist at this point. If it does not exist, you can register with `register()`, an oversimplified limited-scope method which creates container-based job definitions with the `"awslogs"` log driver (for CloudWatch).^[The log group supplied to `crew_aws_batch_monitor()` must be valid. The default is `"/aws/batch/log"`, which may not exist if your system administrator has a custom logging policy.] Below, your container image can be as simple as a Docker Hub identifier (like `"alpine:latest:`) or a full URI of an ECR image.^[For the `crew` controller, you will definitely want an image with R and `crew` installed. For the purposes of testing the monitor, `"alpine:latest"` will work.]
The job definition may or may not exist at this point. If it does not exist, you can register with `register()`, an oversimplified limited-scope method which creates container-based job definitions with the `"awslogs"` log driver (for CloudWatch).^[The log group supplied to `crew_monitor_aws_batch()` must be valid. The default is `"/aws/batch/log"`, which may not exist if your system administrator has a custom logging policy.] Below, your container image can be as simple as a Docker Hub identifier (like `"alpine:latest:`) or a full URI of an ECR image.^[For the `crew` controller, you will definitely want an image with R and `crew` installed. For the purposes of testing the monitor, `"alpine:latest"` will work.]

```{r}
monitor$register(
Expand Down Expand Up @@ -140,7 +140,7 @@ monitor$status(id = job2$id)
#> 1 job2 c38d55ad-4a86-43… arn:… runnable NA 1.70e12 NA NA
```

The `jobs()` method gets the status of all the jobs within the job queue and job definition you originally supplied to `crew_aws_batch_monitor()`. This may include many more jobs than the ones you submitted during the life cycle of the current `monitor` object.
The `jobs()` method gets the status of all the jobs within the job queue and job definition you originally supplied to `crew_monitor_aws_batch()`. This may include many more jobs than the ones you submitted during the life cycle of the current `monitor` object.

```{r}
monitor$jobs()
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,11 +119,11 @@ AWS Batch jobs manually.

`crew.aws.batch` defines a “monitor” class to help you take control of
jobs and job definitions. Create a monitor object with
`crew_aws_batch_monitor()`. You will need to supply a job definition
`crew_monitor_aws_batch()`. You will need to supply a job definition
name and a job queue name.

``` r
monitor <- crew_aws_batch_monitor(
monitor <- crew_monitor_aws_batch(
job_definition = "YOUR_JOB_DEFINITION_NAME",
job_queue = "YOUR_JOB_QUEUE_NAME"
)
Expand Down Expand Up @@ -171,7 +171,7 @@ monitor$status(id = job2$id)

The `jobs()` method gets the status of all the jobs within the job queue
and job definition you originally supplied to
`crew_aws_batch_monitor()`. This may include many more jobs than the
`crew_monitor_aws_batch()`. This may include many more jobs than the
ones you submitted during the life cycle of the current `monitor`
object.

Expand Down Expand Up @@ -362,7 +362,7 @@ https://github.com/wlandau/crew.aws.batch},
`tls = crew_tls(mode = "automatic")`). Please understand and comply
with all the security policies of your organization.

[^3]: The log group supplied to `crew_aws_batch_monitor()` must be
[^3]: The log group supplied to `crew_monitor_aws_batch()` must be
valid. The default is `"/aws/batch/log"`, which may not exist if
your system administrator has a custom logging policy.

Expand Down
8 changes: 4 additions & 4 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ reference:
- title: Package help
contents:
- '`crew.aws.batch-package`'
- title: AWS Batch plugin
- title: AWS Batch launcher plugin
contents:
- '`crew_controller_aws_batch`'
- '`crew_launcher_aws_batch`'
- '`crew_class_launcher_aws_batch`'
- title: Managing AWS Batch jobs and job definitions
- title: AWS Batch monitor
contents:
- '`crew_aws_batch_monitor`'
- '`crew_class_aws_batch_monitor`'
- '`crew_monitor_aws_batch`'
- '`crew_class_monitor_aws_batch`'

0 comments on commit f7f6481

Please sign in to comment.