Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nomad Actions #18794

Merged
merged 8 commits into from
Oct 20, 2023
Merged

Nomad Actions #18794

merged 8 commits into from
Oct 20, 2023

Conversation

philrenaud
Copy link
Contributor

@philrenaud philrenaud commented Oct 18, 2023

Introduces the concept of Actions to Nomad: operator-defined commands and workflows that Nomad users can run on an allocation without having to sweat the details.

These take a similar form to task config blocks and live at task level in the jobspec. To demonstrate:

job "actions-demo" {
  type = "system"

  group "group" {
    task "task" {
      driver = "raw_exec"
      config {
        command = "node"
        args    = ["-e",
        <<EOT
console.log('Howdy!');
// sleep every 3 seconds
setInterval((a) => {
  console.log('Howdy after 3 seconds');
}, 3000);
EOT
        ]
      }

      action "weather" {
        command = "/usr/bin/curl"
        args = ["-s", "wttr.in/Toronto?format=3"]
      }
      action "hello" {
        command = "say"
        args = ["howdy"]
      }
      action "get job info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad job status ${NOMAD_JOB_NAME}
          EOT
        ]
      }
      action "get alloc info" {
        command = "/bin/sh"
        args    = ["-c",
          <<EOT
          nomad alloc status ${NOMAD_ALLOC_ID}
          EOT
        ]
      }

      action "echo time" {
        command = "/bin/sh"
        args    = ["-c", "counter=0; while true; do echo \"Running for ${counter} seconds\"; counter=$((counter + 1)); sleep 1; done"]
      }

    }
  }
}

These actions can be run from the CLI with nomad action by passing in either the allocation ID, or if none is provided, the task and group names and an allocation will be randomly selected:

nomad action \
  -allocation=cafee3ae \
  -job=actions-demo \
weather
nomad action \
  -group=group \
  -task=task \
  -job=actions-demo \
"weather"

An endpoint at v1/job/:id/actions will GET return a list of actions the job has access to, and a GET at v1/job/:id/action?action=NAME&allocID=ID&group=GROUP&task=TASK&tty=true will open a websocket with base64-encoded output (be sure to use Upgrade: websocket and related headers)

Future work:

When #18793 lands, the Nomad web UI will display interactive elements to run actions and view their output in various places:

image
image
image

Further, part of the intent behind actions is to give operators greater ability to specify which sorts of things a Nomad user can perform. With this in mind, ACL changes with actions in mind are planned in the near future.

Resolves #18627

@philrenaud philrenaud self-assigned this Oct 18, 2023
@philrenaud philrenaud linked an issue Oct 18, 2023 that may be closed by this pull request
32 tasks
philrenaud and others added 3 commits October 18, 2023 09:52
* Task-level actions for job submissions and retrieval

* FIXME: Temporary workaround to get ember dev server to pass exec through to 4646

* Update api/tasks.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Update command/agent/job_endpoint.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Diff and copy implementations

* Action structs get their own file, diff updates to behave like our other diffs

* Test to observe actions changes in a version update

* Tests migrated into structs/diff_test and modified with PR comments in mind

* APIActionToSTructsAction now returns a new value

* de-comment some plain parts, remove unused action lookup

* unused param in action converter

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
* unused param in action converter

* backing out of parse_job level and moved toward new endpoint level

* Adds taskName and taskGroupName to actions at job level

* Unmodified job mock actions tests

* actionless job test

* actionless job test

* Multi group multi task actions test

* HTTP method check for GET, cleaner errors in job_endpoint_test

* decomment
@github-actions
Copy link

github-actions bot commented Oct 18, 2023

Ember Test Audit comparison

main 1fe0e11 change
passes 1533 1534 +1
failures 1 1 0
flaky 0 0 0
duration 000ms 000ms -000ms

* Working demo for review purposes

* removal of cors passthru for websockets

* Remove job_endpoint-specific ws handlers and aimed at existing alloc exec handlers instead

* PR comments adressed, no need for taskGroup pass, better group and task lookups from alloc

* early return in action validate and removed jobid from req args per PR comments

* todo removal, we're checking later in the rpc

* boolean style change on tty
* Action command init and stuck-notes

* Conditional reqpath to aim at Job action endpoint

* De-logged

* General CLI command cleanup, observe namespace, pass action as string, get random alloc w group adherence

* tab and varname cleanup

* Remove action param from Allocations().Exec calls
Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the ACL check comment is fixed.

nomad/job_endpoint.go Outdated Show resolved Hide resolved
@kaspergrubbe
Copy link

This is amazing, does it support giving the actions any form of input?

@philrenaud
Copy link
Contributor Author

@kaspergrubbe We've been toying around with the idea internally, and I'd be interested in hearing your use-case. My own feeling on the matter is that allowing operators to provide variable input for actions would create some degree of risk and make the action workflows/output less "guaranteed".

Certainly it would expand functionality and open up some cool new stuff, but we want to weigh that against the potential of providing a foot-gun here.

@benvanstaveren
Copy link

A late comment, but @philrenaud, I want my foot gun. I've got a few cases where being able to enter arguments to an action would be a godsend (even if triggered through the API only); perhaps cage it behind a configuration flag that needs to be explicitly enabled to allow it so that it can be decorated with the usual "warning, foot gun" disclaimers in the docs?

nvanthao pushed a commit to nvanthao/nomad that referenced this pull request Mar 1, 2024
* Scaffolding actions (hashicorp#18639)

* Task-level actions for job submissions and retrieval

* FIXME: Temporary workaround to get ember dev server to pass exec through to 4646

* Update api/tasks.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Update command/agent/job_endpoint.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Diff and copy implementations

* Action structs get their own file, diff updates to behave like our other diffs

* Test to observe actions changes in a version update

* Tests migrated into structs/diff_test and modified with PR comments in mind

* APIActionToSTructsAction now returns a new value

* de-comment some plain parts, remove unused action lookup

* unused param in action converter

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* New endpoint: job/:id/actions (hashicorp#18690)

* unused param in action converter

* backing out of parse_job level and moved toward new endpoint level

* Adds taskName and taskGroupName to actions at job level

* Unmodified job mock actions tests

* actionless job test

* actionless job test

* Multi group multi task actions test

* HTTP method check for GET, cleaner errors in job_endpoint_test

* decomment

* Actions aggregated at job model level (hashicorp#18733)

* Removal of temporary fix to proxy to 4646

* Run Action websocket endpoint (hashicorp#18760)

* Working demo for review purposes

* removal of cors passthru for websockets

* Remove job_endpoint-specific ws handlers and aimed at existing alloc exec handlers instead

* PR comments adressed, no need for taskGroup pass, better group and task lookups from alloc

* early return in action validate and removed jobid from req args per PR comments

* todo removal, we're checking later in the rpc

* boolean style change on tty

* Action CLI command (hashicorp#18778)

* Action command init and stuck-notes

* Conditional reqpath to aim at Job action endpoint

* De-logged

* General CLI command cleanup, observe namespace, pass action as string, get random alloc w group adherence

* tab and varname cleanup

* Remove action param from Allocations().Exec calls

* changelog

* dont nil-check acl

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
nvanthao pushed a commit to nvanthao/nomad that referenced this pull request Mar 1, 2024
* Scaffolding actions (hashicorp#18639)

* Task-level actions for job submissions and retrieval

* FIXME: Temporary workaround to get ember dev server to pass exec through to 4646

* Update api/tasks.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Update command/agent/job_endpoint.go

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Diff and copy implementations

* Action structs get their own file, diff updates to behave like our other diffs

* Test to observe actions changes in a version update

* Tests migrated into structs/diff_test and modified with PR comments in mind

* APIActionToSTructsAction now returns a new value

* de-comment some plain parts, remove unused action lookup

* unused param in action converter

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* New endpoint: job/:id/actions (hashicorp#18690)

* unused param in action converter

* backing out of parse_job level and moved toward new endpoint level

* Adds taskName and taskGroupName to actions at job level

* Unmodified job mock actions tests

* actionless job test

* actionless job test

* Multi group multi task actions test

* HTTP method check for GET, cleaner errors in job_endpoint_test

* decomment

* Actions aggregated at job model level (hashicorp#18733)

* Removal of temporary fix to proxy to 4646

* Run Action websocket endpoint (hashicorp#18760)

* Working demo for review purposes

* removal of cors passthru for websockets

* Remove job_endpoint-specific ws handlers and aimed at existing alloc exec handlers instead

* PR comments adressed, no need for taskGroup pass, better group and task lookups from alloc

* early return in action validate and removed jobid from req args per PR comments

* todo removal, we're checking later in the rpc

* boolean style change on tty

* Action CLI command (hashicorp#18778)

* Action command init and stuck-notes

* Conditional reqpath to aim at Job action endpoint

* De-logged

* General CLI command cleanup, observe namespace, pass action as string, get random alloc w group adherence

* tab and varname cleanup

* Remove action param from Allocations().Exec calls

* changelog

* dont nil-check acl

---------

Co-authored-by: Tim Gross <tgross@hashicorp.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

☂️ Nomad Actions
4 participants